diff mbox series

[ovs-dev,v2] Add support for centralize routing for distributed gw ports.

Message ID 20240730023850.1671255-1-numans@ovn.org
State Accepted
Headers show
Series [ovs-dev,v2] Add support for centralize routing for distributed gw ports. | expand

Checks

Context Check Description
ovsrobot/apply-robot success apply and check: success
ovsrobot/github-robot-_Build_and_Test success github build: passed
ovsrobot/github-robot-_ovn-kubernetes success github build: passed

Commit Message

Numan Siddique July 30, 2024, 2:38 a.m. UTC 
From: Numan Siddique <numans@ovn.org>

Consider a deployment with the below logical resources:

1. A bridged logical switch 'public' with a port - P1 and a localnet
   port ln-public.
2. A logical router 'R'
3. Logical switch 'public' connected to R via logical switch/router port
   peers (public-R and R-public).
4. R-public is distributed gateway port with its network as 172.16.0.0/24
5. NATs (dnat_and_snat) configured in 'R'.
6. And a few overlay logical switches S1, S2 to R.

Any traffic from logical port - P1 of public logical switch destined to
S1 or S2's logical ports goes out of the source chassis
(where P1 resides) via the localnet port and reaches the gateway chassis
which handles the routing.

There are couple of traffic flow scenarios which doesn't work if the
logical switch 'public' doesn't have a localnet port.

1. Traffic from port - P1 destined to logical switches S1 or S2 gets
   dropped in the source chassis.  The packet enters the router R's
   pipeline, but it gets dropped in the 'lr_in_admission' stage since
   the logical flow to allow traffic destined to the distributed gateway
   port MAC is installed only on the gateway chassis.

2. NAT doesn't work as expected.

In order to suppose this use case (of a logical switch not having a
localnet port, but has a distributed gateway port and NATs), this patch
supports the option 'centralize_routing', which can be configured on
the distributed gateway port (R-public in the example above).
If this option is set, then routing is centralized on the gateway
chassis for the traffic destined to the R-public's networks
(172.16.0.0/24 for the above example).  Traffic from P1 will be
tunnelled to the gateway chassis.

ovn-northd creates a chassisresident port (cr-public-R) for the
logical switch port - public-R, along with cr-R-public inorder to
centralize the traffic.

This feature gets enabled for the distributed gateway port R-public if
  - The above option is set to true in the R-public's options column.
  - The logical switch 'public' doesn't have any localnet ports.
  - And R-public is the only distributed gateway port of R.

Distributed NAT (i.e if external_mac and router_port is set) is
not supported and instead the router port mac is used for such traffic
and centralized on the gateway chassis.

Reported-at: https://issues.redhat.com/browse/FDP-364
Acked-by: Mark Michelson <mmichels@redhat.com>
Signed-off-by: Numan Siddique <numans@ovn.org>
---

v1 -> v2
-------
   * Corrected the NEWS item entry for the new option.
   * Rebased and resolved conflicts.

Note: This patch is the continuation from this series - https://patchwork.ozlabs.org/project/ovn/patch/20240606214432.168750-1-numans@ovn.org/
      Resetted the version number since the new option changed from LSP
      to LRP.


 NEWS                      |   3 +
 controller/physical.c     |   4 +
 northd/northd.c           | 257 +++++++++++++++----
 northd/northd.h           |   1 +
 ovn-nb.xml                |  34 +++
 tests/multinode-macros.at |   2 +-
 tests/multinode.at        | 177 +++++++++++++
 tests/ovn-northd.at       | 516 +++++++++++++++++++++++++++++++++++++-
 tests/ovn.at              |   8 +-
 9 files changed, 952 insertions(+), 50 deletions(-)

Comments

Mark Michelson Aug. 2, 2024, 6:10 p.m. UTC | #1 
I know my Ack is already on this, but I had another look, and just to 
reiterate:

Acked-by: Mark Michelson <mmichels@redhat.com>

On 7/29/24 22:38, numans@ovn.org wrote:
> From: Numan Siddique <numans@ovn.org>
> 
> Consider a deployment with the below logical resources:
> 
> 1. A bridged logical switch 'public' with a port - P1 and a localnet
>     port ln-public.
> 2. A logical router 'R'
> 3. Logical switch 'public' connected to R via logical switch/router port
>     peers (public-R and R-public).
> 4. R-public is distributed gateway port with its network as 172.16.0.0/24
> 5. NATs (dnat_and_snat) configured in 'R'.
> 6. And a few overlay logical switches S1, S2 to R.
> 
> Any traffic from logical port - P1 of public logical switch destined to
> S1 or S2's logical ports goes out of the source chassis
> (where P1 resides) via the localnet port and reaches the gateway chassis
> which handles the routing.
> 
> There are couple of traffic flow scenarios which doesn't work if the
> logical switch 'public' doesn't have a localnet port.
> 
> 1. Traffic from port - P1 destined to logical switches S1 or S2 gets
>     dropped in the source chassis.  The packet enters the router R's
>     pipeline, but it gets dropped in the 'lr_in_admission' stage since
>     the logical flow to allow traffic destined to the distributed gateway
>     port MAC is installed only on the gateway chassis.
> 
> 2. NAT doesn't work as expected.
> 
> In order to suppose this use case (of a logical switch not having a
> localnet port, but has a distributed gateway port and NATs), this patch
> supports the option 'centralize_routing', which can be configured on
> the distributed gateway port (R-public in the example above).
> If this option is set, then routing is centralized on the gateway
> chassis for the traffic destined to the R-public's networks
> (172.16.0.0/24 for the above example).  Traffic from P1 will be
> tunnelled to the gateway chassis.
> 
> ovn-northd creates a chassisresident port (cr-public-R) for the
> logical switch port - public-R, along with cr-R-public inorder to
> centralize the traffic.
> 
> This feature gets enabled for the distributed gateway port R-public if
>    - The above option is set to true in the R-public's options column.
>    - The logical switch 'public' doesn't have any localnet ports.
>    - And R-public is the only distributed gateway port of R.
> 
> Distributed NAT (i.e if external_mac and router_port is set) is
> not supported and instead the router port mac is used for such traffic
> and centralized on the gateway chassis.
> 
> Reported-at: https://issues.redhat.com/browse/FDP-364
> Acked-by: Mark Michelson <mmichels@redhat.com>
> Signed-off-by: Numan Siddique <numans@ovn.org>
> ---
> 
> v1 -> v2
> -------
>     * Corrected the NEWS item entry for the new option.
>     * Rebased and resolved conflicts.
> 
> Note: This patch is the continuation from this series - https://patchwork.ozlabs.org/project/ovn/patch/20240606214432.168750-1-numans@ovn.org/
>        Resetted the version number since the new option changed from LSP
>        to LRP.
> 
> 
>   NEWS                      |   3 +
>   controller/physical.c     |   4 +
>   northd/northd.c           | 257 +++++++++++++++----
>   northd/northd.h           |   1 +
>   ovn-nb.xml                |  34 +++
>   tests/multinode-macros.at |   2 +-
>   tests/multinode.at        | 177 +++++++++++++
>   tests/ovn-northd.at       | 516 +++++++++++++++++++++++++++++++++++++-
>   tests/ovn.at              |   8 +-
>   9 files changed, 952 insertions(+), 50 deletions(-)
> 
> diff --git a/NEWS b/NEWS
> index 87e326f21e..8440a74677 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -47,6 +47,9 @@ Post v24.03.0
>     - Add support for CT zone limit that can be specified per LR
>       (options:ct-zone-limit), LS (other_config:ct-zone-limit) or LSP
>       (options:ct-zone-limit).
> +  - A new LRP option 'centralize_routing' has been added to a
> +    distributed gateway port to centralize routing if the logical
> +    switch of its peer doesn't have a localnet port.
>   
>   OVN v24.03.0 - 01 Mar 2024
>   --------------------------
> diff --git a/controller/physical.c b/controller/physical.c
> index 3c0200c383..9e04ad5f22 100644
> --- a/controller/physical.c
> +++ b/controller/physical.c
> @@ -1610,6 +1610,10 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name,
>                                                       ct_zones);
>               put_zones_ofpacts(&zone_ids, ofpacts_p);
>   
> +            /* Clear the MFF_INPORT.  Its possible that the same packet may
> +             * go out from the same tunnel inport. */
> +            put_load(ofp_to_u16(OFPP_NONE), MFF_IN_PORT, 0, 16, ofpacts_p);
> +
>               /* Resubmit to table 41. */
>               put_resubmit(OFTABLE_CHECK_LOOPBACK, ofpacts_p);
>           }
> diff --git a/northd/northd.c b/northd/northd.c
> index 5c2fd74ff1..4f59d4f1a3 100644
> --- a/northd/northd.c
> +++ b/northd/northd.c
> @@ -2107,6 +2107,55 @@ parse_lsp_addrs(struct ovn_port *op)
>       }
>   }
>   
> +static struct ovn_port *
> +create_cr_port(struct ovn_port *op, struct hmap *ports,
> +               struct ovs_list *both_dbs, struct ovs_list *nb_only)
> +{
> +    char *redirect_name = ovn_chassis_redirect_name(
> +        op->nbsp ? op->nbsp->name : op->nbrp->name);
> +
> +    struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> +    if (crp && crp->sb && crp->sb->datapath == op->od->sb) {
> +        ovn_port_set_nb(crp, NULL, op->nbrp);
> +        ovs_list_remove(&crp->list);
> +        ovs_list_push_back(both_dbs, &crp->list);
> +    } else {
> +        crp = ovn_port_create(ports, redirect_name,
> +                              op->nbsp, op->nbrp, NULL);
> +        ovs_list_push_back(nb_only, &crp->list);
> +    }
> +
> +    crp->primary_port = op;
> +    op->cr_port = crp;
> +    crp->od = op->od;
> +    free(redirect_name);
> +
> +    return crp;
> +}
> +
> +/* Returns true if chassis resident port needs to be created for
> + * op's peer logical switch.  False otherwise.
> + *
> + * Chassis resident port needs to be created if the following
> + * conditionsd are met:
> + *   - op is a distributed gateway port
> + *   - op has the option 'centralize_routing' set to true
> + *   - op is the only distributed gateway port attached to its
> + *     router
> + *   - op's peer logical switch has no localnet ports.
> + */
> +static bool
> +peer_needs_cr_port_creation(struct ovn_port *op)
> +{
> +    if ((op->nbrp->n_gateway_chassis || op->nbrp->ha_chassis_group)
> +        && op->od->n_l3dgw_ports == 1 && op->peer && op->peer->nbsp
> +        && !op->peer->od->n_localnet_ports) {
> +        return smap_get_bool(&op->nbrp->options, "centralize_routing", false);
> +    }
> +
> +    return false;
> +}
> +
>   static void
>   join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                      struct hmap *ls_datapaths, struct hmap *lr_datapaths,
> @@ -2214,9 +2263,10 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>               tag_alloc_add_existing_tags(tag_alloc_table, nbsp);
>           }
>       }
> +
> +    struct hmapx dgps = HMAPX_INITIALIZER(&dgps);
>       HMAP_FOR_EACH (od, key_node, lr_datapaths) {
>           ovs_assert(od->nbr);
> -        size_t n_allocated_l3dgw_ports = 0;
>           for (size_t i = 0; i < od->nbr->n_ports; i++) {
>               const struct nbrec_logical_router_port *nbrp
>                   = od->nbr->ports[i];
> @@ -2280,10 +2330,7 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                       redirect_type && !strcasecmp(redirect_type, "bridged");
>               }
>   
> -            if (op->nbrp->ha_chassis_group ||
> -                op->nbrp->n_gateway_chassis) {
> -                /* Additional "derived" ovn_port crp represents the
> -                 * instance of op on the gateway chassis. */
> +            if (op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis) {
>                   const char *gw_chassis = smap_get(&op->od->nbr->options,
>                                                  "chassis");
>                   if (gw_chassis) {
> @@ -2292,34 +2339,9 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                       VLOG_WARN_RL(&rl, "Bad configuration: distributed "
>                                    "gateway port configured on port %s "
>                                    "on L3 gateway router", nbrp->name);
> -                    continue;
> -                }
> -
> -                char *redirect_name =
> -                    ovn_chassis_redirect_name(nbrp->name);
> -                struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> -                if (crp && crp->sb && crp->sb->datapath == od->sb) {
> -                    ovn_port_set_nb(crp, NULL, nbrp);
> -                    ovs_list_remove(&crp->list);
> -                    ovs_list_push_back(both, &crp->list);
>                   } else {
> -                    crp = ovn_port_create(ports, redirect_name,
> -                                          NULL, nbrp, NULL);
> -                    ovs_list_push_back(nb_only, &crp->list);
> -                }
> -                crp->primary_port = op;
> -                op->cr_port = crp;
> -                crp->od = od;
> -                free(redirect_name);
> -
> -                /* Add to l3dgw_ports in od, for later use during flow
> -                 * creation. */
> -                if (od->n_l3dgw_ports == n_allocated_l3dgw_ports) {
> -                    od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> -                                                 &n_allocated_l3dgw_ports,
> -                                                 sizeof *od->l3dgw_ports);
> +                    hmapx_add(&dgps, op);
>                   }
> -                od->l3dgw_ports[od->n_l3dgw_ports++] = op;
>              }
>           }
>       }
> @@ -2376,12 +2398,6 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                           arp_proxy, op->nbsp->name);
>                   }
>               }
> -
> -            /* Only used for the router type LSP whose peer is l3dgw_port */
> -            if (op->peer && is_l3dgw_port(op->peer)) {
> -                op->enable_router_port_acl = smap_get_bool(
> -                    &op->nbsp->options, "enable_router_port_acl", false);
> -            }
>           } else if (op->nbrp && op->nbrp->peer && !is_cr_port(op)) {
>               struct ovn_port *peer = ovn_port_find(ports, op->nbrp->peer);
>               if (peer) {
> @@ -2402,6 +2418,57 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>           }
>       }
>   
> +    struct hmapx_node *hmapx_node;
> +    HMAPX_FOR_EACH (hmapx_node, &dgps) {
> +        op = hmapx_node->data;
> +        od = op->od;
> +        ovs_assert(op->nbrp);
> +        ovs_assert(op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis);
> +
> +        /* Additional "derived" ovn_port crp represents the instance of op on
> +         * the gateway chassis. */
> +        struct ovn_port *crp = create_cr_port(op, ports, both, nb_only);
> +        ovs_assert(crp);
> +
> +        /* Add to l3dgw_ports in od, for later use during flow creation. */
> +        if (od->n_l3dgw_ports == od->n_allocated_l3dgw_ports) {
> +            od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> +                                        &od->n_allocated_l3dgw_ports,
> +                                        sizeof *od->l3dgw_ports);
> +        }
> +        od->l3dgw_ports[od->n_l3dgw_ports++] = op;
> +
> +        if (op->peer && op->peer->nbsp) {
> +            /* Only used for the router type LSP whose peer is l3dgw_port */
> +            op->peer->enable_router_port_acl = smap_get_bool(
> +                    &op->peer->nbsp->options, "enable_router_port_acl", false);
> +        }
> +    }
> +
> +
> +    /* Create chassisresident port for the distributed gateway port's (DGP)
> +     * peer if
> +     *  - DGP's router has only one DGP and
> +     *  - Its peer is a logical switch port and
> +     *  - It's peer's logical switch has no localnet ports and
> +     *  - option 'centralize_routing' is set to true for the DGP.
> +     *
> +     * This is required to support
> +     *   - NAT via geneve (for the overlay provider networks) and
> +     *   - to centralize routing on the gateway chassis for the traffic
> +     *     destined to the DGP's networks.
> +     *
> +     * Future enhancement: Support 'centralizerouting' for all the DGP's
> +     * of a logical router.
> +     * */
> +    HMAPX_FOR_EACH (hmapx_node, &dgps) {
> +        op = hmapx_node->data;
> +        if (peer_needs_cr_port_creation(op)) {
> +            create_cr_port(op->peer, ports, both, nb_only);
> +        }
> +    }
> +    hmapx_destroy(&dgps);
> +
>       /* Wait until all ports have been connected to add to IPAM since
>        * it relies on proper peers to be set
>        */
> @@ -3184,16 +3251,28 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn,
>                * type "l3gateway". */
>               if (chassis) {
>                   sbrec_port_binding_set_type(op->sb, "l3gateway");
> +            } else if (is_cr_port(op)) {
> +                sbrec_port_binding_set_type(op->sb, "chassisredirect");
> +                ovs_assert(op->primary_port->peer);
> +                ovs_assert(op->primary_port->peer->cr_port);
> +                ovs_assert(op->primary_port->peer->cr_port->sb);
> +                sbrec_port_binding_set_ha_chassis_group(
> +                    op->sb,
> +                    op->primary_port->peer->cr_port->sb->ha_chassis_group);
> +
>               } else {
>                   sbrec_port_binding_set_type(op->sb, "patch");
>               }
>   
>               const char *router_port = smap_get(&op->nbsp->options,
>                                                  "router-port");
> -            if (router_port || chassis) {
> +            if (router_port || chassis || is_cr_port(op)) {
>                   struct smap new;
>                   smap_init(&new);
> -                if (router_port) {
> +
> +                if (is_cr_port(op)) {
> +                    smap_add(&new, "distributed-port", op->nbsp->name);
> +                } else if (router_port) {
>                       smap_add(&new, "peer", router_port);
>                   }
>                   if (chassis) {
> @@ -8155,9 +8234,27 @@ build_lswitch_rport_arp_req_flow(
>       struct lflow_ref *lflow_ref)
>   {
>       struct ds match   = DS_EMPTY_INITIALIZER;
> +    struct ds m       = DS_EMPTY_INITIALIZER;
>       struct ds actions = DS_EMPTY_INITIALIZER;
>   
> -    arp_nd_ns_match(ips, addr_family, &match);
> +    arp_nd_ns_match(ips, addr_family, &m);
> +    ds_clone(&match, &m);
> +
> +    bool has_cr_port = patch_op->cr_port;
> +
> +    /* If the patch_op has a chassis resident port, it means
> +     *    - its peer is a distributed gateway port (DGP) and
> +     *    - routing is centralized for the DGP's networks on
> +     *      the configured gateway chassis.
> +     *
> +     * If that's the case, make sure that the packets destined to
> +     * the DGP's MAC are sent to the chassis where the DGP resides.
> +     * */
> +
> +    if (has_cr_port) {
> +        ds_put_format(&match, " && is_chassis_resident(%s)",
> +                      patch_op->cr_port->json_key);
> +    }
>   
>       /* Send a the packet to the router pipeline.  If the switch has non-router
>        * ports then flood it there as well.
> @@ -8179,6 +8276,31 @@ build_lswitch_rport_arp_req_flow(
>                                   lflow_ref);
>       }
>   
> +    if (has_cr_port) {
> +        ds_clear(&match);
> +        ds_put_format(&match, "%s && !is_chassis_resident(%s)", ds_cstr(&m),
> +                      patch_op->cr_port->json_key);
> +        ds_clear(&actions);
> +        if (od->n_router_ports != od->nbs->n_ports) {
> +            ds_put_format(&actions, "clone {outport = %s; output; }; "
> +                                    "outport = \""MC_FLOOD_L2"\"; output;",
> +                          patch_op->cr_port->json_key);
> +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> +                                    priority, ds_cstr(&match),
> +                                    ds_cstr(&actions), stage_hint,
> +                                    lflow_ref);
> +        } else {
> +            ds_put_format(&actions, "outport = %s; output;",
> +                          patch_op->cr_port->json_key);
> +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> +                                    priority, ds_cstr(&match),
> +                                    ds_cstr(&actions),
> +                                    stage_hint,
> +                                    lflow_ref);
> +        }
> +    }
> +
> +    ds_destroy(&m);
>       ds_destroy(&match);
>       ds_destroy(&actions);
>   }
> @@ -9548,7 +9670,11 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
>                                   struct ds *actions, struct ds *match)
>   {
>       ovs_assert(op->nbsp);
> -    if (lsp_is_external(op->nbsp)) {
> +
> +    /* Note: A switch port can also have a chassis resident derived port.
> +     * Check if 'op' is a chassis resident dervied port. If so, skip
> +     * adding unicast lookup flows for this port. */
> +    if (lsp_is_external(op->nbsp) || is_cr_port(op)) {
>           return;
>       }
>   
> @@ -9566,8 +9692,6 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
>                              "outport = \""MC_UNKNOWN "\"; output;"
>                            : "outport = %s; output;")
>                            : debug_drop_action();
> -    ds_clear(actions);
> -    ds_put_format(actions, action, op->json_key);
>   
>       if (lsp_is_router(op->nbsp) && op->peer && op->peer->nbrp) {
>           /* For ports connected to logical routers add flows to bypass the
> @@ -9614,14 +9738,43 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
>               if (add_chassis_resident_check) {
>                   ds_put_format(match, " && is_chassis_resident(%s)", json_key);
>               }
> +        } else if (op->cr_port) {
> +            /* If the op has a chassis resident port, it means
> +             *   - its peer is a distributed gateway port (DGP) and
> +             *   - routing is centralized for the DGP's networks on
> +             *     the configured gateway chassis.
> +             *
> +             * If that's the case, make sure that the packets destined to
> +             * the DGP's MAC are sent to the chassis where the DGP resides.
> +             * */
> +            ds_clear(actions);
> +            ds_put_format(actions, action, op->cr_port->json_key);
> +
> +            struct ds m = DS_EMPTY_INITIALIZER;
> +            ds_put_format(&m, "eth.dst == %s && !is_chassis_resident(%s)",
> +                          op->peer->lrp_networks.ea_s,
> +                          op->cr_port->json_key);
> +
> +            ovn_lflow_add_with_hint(lflows, op->od,
> +                                    S_SWITCH_IN_L2_LKUP, 50,
> +                                    ds_cstr(&m), ds_cstr(actions),
> +                                    &op->nbsp->header_,
> +                                    op->lflow_ref);
> +            ds_destroy(&m);
> +            ds_put_format(match, " && is_chassis_resident(%s)",
> +                          op->cr_port->json_key);
>           }
>   
> +        ds_clear(actions);
> +        ds_put_format(actions, action, op->json_key);
>           ovn_lflow_add_with_hint(lflows, op->od,
>                                   S_SWITCH_IN_L2_LKUP, 50,
>                                   ds_cstr(match), ds_cstr(actions),
>                                   &op->nbsp->header_,
>                                   op->lflow_ref);
>       } else {
> +        ds_clear(actions);
> +        ds_put_format(actions, action, op->json_key);
>           for (size_t i = 0; i < op->n_lsp_addrs; i++) {
>               ds_clear(match);
>               ds_put_format(match, "eth.dst == %s", op->lsp_addrs[i].ea_s);
> @@ -11725,6 +11878,14 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op,
>           return;
>       }
>   
> +    if (op->peer && op->peer->cr_port) {
> +        /* We don't add the below flows if the router port's peer has
> +         * a chassisresident port.  That's because routing is centralized on
> +         * the gateway chassis for the router port networks/subnets.
> +         */
> +        return;
> +    }
> +
>       /* Mac address to use when replying to ARP/NS. */
>       const char *mac_s = REG_INPORT_ETH_ADDR;
>       struct eth_addr mac;
> @@ -15109,6 +15270,16 @@ lrouter_check_nat_entry(const struct ovn_datapath *od,
>       /* For distributed router NAT, determine whether this NAT rule
>        * satisfies the conditions for distributed NAT processing. */
>       *distributed = false;
> +
> +    /* NAT cannnot be distributed if the DGP's peer
> +     * has a chassisresident port (as the routing is centralized
> +     * on the gateway chassis for the DGP's networks/subnets.)
> +     */
> +    struct ovn_port *l3dgw_port = *nat_l3dgw_port;
> +    if (l3dgw_port && l3dgw_port->peer && l3dgw_port->peer->cr_port) {
> +        return 0;
> +    }
> +
>       if (od->n_l3dgw_ports && !strcmp(nat->type, "dnat_and_snat") &&
>           nat->logical_port && nat->external_mac) {
>           if (eth_addr_from_string(nat->external_mac, mac)) {
> diff --git a/northd/northd.h b/northd/northd.h
> index d4a8d75abc..d7c9655916 100644
> --- a/northd/northd.h
> +++ b/northd/northd.h
> @@ -325,6 +325,7 @@ struct ovn_datapath {
>        * will be NULL. */
>       struct ovn_port **l3dgw_ports;
>       size_t n_l3dgw_ports;
> +    size_t n_allocated_l3dgw_ports;
>   
>       /* router datapath has a logical port with redirect-type set to bridged. */
>       bool redirect_bridged;
> diff --git a/ovn-nb.xml b/ovn-nb.xml
> index a4362a4ef1..217d1cd1fe 100644
> --- a/ovn-nb.xml
> +++ b/ovn-nb.xml
> @@ -3499,6 +3499,40 @@ or
>             <ref column="options" key="gateway_mtu"/> option.
>           </p>
>         </column>
> +
> +      <column name="options" key="centralize_routing"
> +              type='{"type": "boolean"}'>
> +        <p>
> +          This option is applicable only if the router port is a
> +          distributed gateway port i.e if the <ref table="Logical_Router_Port"
> +          column="ha_chassis_group"/> column or
> +          <ref table="Logical_Router_Port" column="gateway_chassis"/>
> +          is set.
> +        </p>
> +
> +        <p>
> +          If set to <code>true</code>, routing for the router port's
> +          networks (set in the column <ref table="Logical_Router_Port"
> +          column="networks"/>) is centralized on the gateway chassis
> +          which claims this distributed gateway port.
> +        </p>
> +
> +        <p>
> +          Additionally for this option to take effect, below conditions
> +          must be met:
> +        </p>
> +
> +        <ul>
> +          <li>
> +            The Logical router has only one distributed gateway port.
> +          </li>
> +
> +          <li>
> +            The router port's peer logical switch has no localnet ports.
> +          </li>
> +
> +        </ul>
> +      </column>
>       </group>
>   
>       <group title="Attachment">
> diff --git a/tests/multinode-macros.at b/tests/multinode-macros.at
> index 786e564860..757917626c 100644
> --- a/tests/multinode-macros.at
> +++ b/tests/multinode-macros.at
> @@ -92,7 +92,7 @@ m_count_rows() {
>   m_check_row_count() {
>       local db=$(parse_db $1) table=$(parse_table $1); shift
>       local count=$1; shift
> -    local found=$(m_count_rows $c $db:$table "$@")
> +    local found=$(m_count_rows $db:$table "$@")
>       echo
>       echo "Checking for $count rows in $db $table${1+ with $*}... found $found"
>       if test "$count" != "$found"; then
> diff --git a/tests/multinode.at b/tests/multinode.at
> index a7231130ac..a725414165 100644
> --- a/tests/multinode.at
> +++ b/tests/multinode.at
> @@ -1033,6 +1033,183 @@ M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [sh -c 'dd bs=512 count=2 if=/dev/uran
>   done
>   M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ip route get 10.0.0.1 dev sw0p1 | grep -q 'mtu 942'])
>   
> +# Reset back to geneve tunnels
> +for c in ovn-chassis-1 ovn-chassis-2 ovn-gw-1
> +do
> +    m_as $c ovs-vsctl set open . external-ids:ovn-encap-type=geneve
> +done
> +
> +AT_CLEANUP
> +
> +AT_SETUP([ovn multinode NAT on a provider network with no localnet ports])
> +
> +# Check that ovn-fake-multinode setup is up and running
> +check_fake_multinode_setup
> +
> +# Delete the multinode NB and OVS resources before starting the test.
> +cleanup_multinode_resources
> +
> +check multinode_nbctl ls-add sw0
> +check multinode_nbctl lsp-add sw0 sw0-port1
> +check multinode_nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03 10.0.0.3 1000::3"
> +check multinode_nbctl lsp-add sw0 sw0-port2
> +check multinode_nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:04 10.0.0.4 1000::4"
> +
> +m_as ovn-chassis-1 /data/create_fake_vm.sh sw0-port1 sw0p1 50:54:00:00:00:03 10.0.0.3 24 10.0.0.1 1000::3/64 1000::a
> +m_as ovn-chassis-2 /data/create_fake_vm.sh sw0-port2 sw0p2 50:54:00:00:00:04 10.0.0.4 24 10.0.0.1 1000::4/64 1000::a
> +
> +m_wait_for_ports_up
> +
> +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.4 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +# Create the second logical switch with one port
> +check multinode_nbctl ls-add sw1
> +check multinode_nbctl lsp-add sw1 sw1-port1
> +check multinode_nbctl lsp-set-addresses sw1-port1 "40:54:00:00:00:03 20.0.0.3 2000::3"
> +
> +# Create a logical router and attach both logical switches
> +check multinode_nbctl lr-add lr0
> +check multinode_nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 1000::a/64
> +check multinode_nbctl lsp-add sw0 sw0-lr0
> +check multinode_nbctl lsp-set-type sw0-lr0 router
> +check multinode_nbctl lsp-set-addresses sw0-lr0 router
> +check multinode_nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> +
> +check multinode_nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:02 20.0.0.1/24 2000::a/64
> +check multinode_nbctl lsp-add sw1 sw1-lr0
> +check multinode_nbctl lsp-set-type sw1-lr0 router
> +check multinode_nbctl lsp-set-addresses sw1-lr0 router
> +check multinode_nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> +
> +m_as ovn-chassis-2 /data/create_fake_vm.sh sw1-port1 sw1p1 40:54:00:00:00:03 20.0.0.3 24 20.0.0.1 2000::3/64 2000::a
> +
> +# create exteranl connection for N/S traffic
> +check multinode_nbctl ls-add public
> +check multinode_nbctl lsp-add public ln-public
> +check multinode_nbctl lsp-set-type ln-public localnet
> +check multinode_nbctl lsp-set-addresses ln-public unknown
> +check multinode_nbctl lsp-set-options ln-public network_name=public
> +
> +check multinode_nbctl lrp-add lr0 lr0-public 00:11:22:00:ff:01 172.20.0.100/24
> +check multinode_nbctl lsp-add public public-lr0
> +check multinode_nbctl lsp-set-type public-lr0 router
> +check multinode_nbctl lsp-set-addresses public-lr0 router
> +check multinode_nbctl lsp-set-options public-lr0 router-port=lr0-public
> +check multinode_nbctl lrp-set-gateway-chassis lr0-public ovn-gw-1 10
> +
> +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
> +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.120 20.0.0.3
> +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 10.0.0.0/24
> +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 20.0.0.0/24
> +
> +# Create a logical port pub-p1 and bind it in ovn-chassis-1
> +check multinode_nbctl lsp-add public public-port1
> +check multinode_nbctl lsp-set-addresses public-port1 "60:54:00:00:00:03 172.168.0.50"
> +
> +m_as ovn-chassis-1 /data/create_fake_vm.sh public-port1 pubp1 60:54:00:00:00:03 172.20.0.50 24 172.20.0.100
> +
> +check multinode_nbctl --wait=hv sync
> +
> +# First do basic ping tests before deleting the localnet port - ln-public.
> +# Once the localnet port is deleted from public ls, routing for 172.20.0.0/24
> +# is centralized on ovn-gw-1.
> +
> +# This function checks the North-South traffic.
> +run_ns_traffic() {
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.110], [ignore], [ignore])
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.120], [ignore], [ignore])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-2], [sw1p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  # Now ping from pubp1 to 172.20.0.100, 172.20.0.110, 172.20.0.120, 10.0.0.3 and 20.0.0.3
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.3 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 20.0.0.3 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +}
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
> +# Delete the localnet port by changing the type of ln-public to VIF port.
> +check multinode_nbctl --wait=hv lsp-set-type ln-public ""
> +
> +# cr-port should not be created for public-lr0 since the option
> +# centralize_routing=true is not yet set for lr0-public.
> +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +# Set the option - centralize_routing now.
> +check multinode_nbctl --wait=hv set logical_router_port lr0-public options:centralize_routing=true
> +
> +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> +m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
> +# Re-add the localnet port
> +check multinode_nbctl --wait=hv lsp-set-type ln-public localnet
> +
> +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
> +# Delete the ln-public port this time.
> +check multinode_nbctl --wait=hv lsp-del ln-public
> +
> +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> +m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
>   AT_CLEANUP
>   
>   AT_SETUP([ovn provider network - always_tunnel])
> diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
> index 57f89a7746..649c20f285 100644
> --- a/tests/ovn-northd.at
> +++ b/tests/ovn-northd.at
> @@ -2187,7 +2187,7 @@ match=(inport == "lrp-public" && arp.op == 1 && arp.tpa == 43.43.43.4 && is_chas
>   action=(eth.dst = eth.src; eth.src = 00:00:00:00:00:02; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 00:00:00:00:00:02; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
>   ])
>   
> -# xreg0[0..47] isn't used anywhere else.
> +# xreg0[[0..47]] isn't used anywhere else.
>   AT_CHECK([ovn-sbctl lflow-list | grep "xreg0\[[0..47\]]" | grep -vE 'lr_in_admission|lr_in_ip_input'], [1], [])
>   
>   AT_CLEANUP
> @@ -5503,13 +5503,14 @@ AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | grep "192.168.4.100" | grep "_MC_flo
>   
>   AS_BOX([Configuring ro1-ls1 router port as a gateway router port])
>   
> -ovn-nbctl --wait=sb lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> +check ovn-nbctl  lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> +check ovn-nbctl --wait=sb lsp-add ls1 ln-ls1 -- lsp-set-type ln-ls1 localnet
>   
>   ovn-sbctl lflow-list ls1 > ls1_lflows
>   AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | ovn_strip_lflows], [0], [dnl
>     table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
>     table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> -  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01), action=(outport = "ls1-ro1"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01 && is_chassis_resident("cr-ro1-ls1")), action=(outport = "ls1-ro1"; output;)
>     table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:02), action=(outport = "vm1"; output;)
>     table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
>     table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:01:01} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> @@ -12475,3 +12476,512 @@ AT_CHECK([ovn-sbctl dump-flows lr | grep lr_in_dnat | ovn_strip_lflows], [0], [d
>   
>   AT_CLEANUP
>   ])
> +
> +OVN_FOR_EACH_NORTHD_NO_HV([
> +AT_SETUP([NAT on a provider network with no localnet ports])
> +AT_KEYWORDS([NAT])
> +ovn_start
> +
> +check ovn-nbctl -- ls-add sw0 -- ls-add sw1
> +check ovn-nbctl lsp-add sw0 sw0-port1
> +check ovn-nbctl lr-add lr0
> +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
> +check ovn-nbctl lsp-add sw0 sw0-lr0
> +check ovn-nbctl lsp-set-type sw0-lr0 router
> +check ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01
> +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> +
> +check ovn-nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:03 20.0.0.1/24
> +check ovn-nbctl lsp-add sw1 sw1-lr0
> +check ovn-nbctl lsp-set-type sw1-lr0 router
> +check ovn-nbctl lsp-set-addresses sw1-lr0 router
> +check ovn-nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> +
> +check ovn-sbctl chassis-add gw1 geneve 127.0.0.1
> +check ovn-nbctl ls-add public
> +check ovn-nbctl lsp-add public pub-p1
> +
> +# localnet port
> +check ovn-nbctl lsp-add public ln-public
> +check ovn-nbctl lsp-set-type ln-public localnet
> +check ovn-nbctl lsp-set-addresses ln-public unknown
> +check ovn-nbctl lsp-set-options ln-public network_name=public
> +
> +check ovn-nbctl lrp-add lr0 lr0-public 00:00:00:00:ff:02 172.168.0.10/24
> +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1
> +
> +check ovn-nbctl lsp-add public public-lr0
> +check ovn-nbctl lsp-set-type public-lr0 router
> +check ovn-nbctl lsp-set-addresses public-lr0 router
> +check ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public
> +
> +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
> +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.120 20.0.0.3
> +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 10.0.0.0/24
> +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 20.0.0.0/24
> +
> +check ovn-nbctl --wait=sb sync
> +
> +check_flows_no_cr_port_for_public_lr0() {
> +  # check that there is no port binding cr-public-lr0
> +  check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +  ovn-sbctl dump-flows lr0 > lr0flows
> +  ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
> +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +])
> +
> +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +])
> +
> +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +])
> +
> +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +])
> +
> +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> +])
> +
> +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +])
> +
> +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +])
> +
> +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-lr0-public")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +}
> +
> +check_flows_cr_port_for_public_lr0() {
> +  # check that there is port binding cr-public-lr0
> +  check_row_count Port_Binding 1 logical_port=cr-public-lr0
> +  check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> +
> +  ovn-sbctl dump-flows lr0 > lr0flows
> +  ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
> +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +])
> +
> +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +])
> +
> +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +])
> +
> +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +])
> +
> +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> +])
> +
> +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +])
> +
> +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +])
> +
> +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-public-lr0")), action=(outport = "cr-public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-public-lr0")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +}
> +
> +# Check that the lflows are as expected when public has localnet port.
> +check_flows_no_cr_port_for_public_lr0
> +
> +# Remove the localnet port from public logical switch.
> +check ovn-nbctl --wait=sb lsp-set-type ln-public ""
> +
> +# Check that the lflows are as expected and there is no cr port
> +# created for "public-lr0"  when public has no localnet port
> +# since public doesn't have the option "overlay_provider_network=true"
> +# set.
> +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +ovn-sbctl dump-flows lr0 > lr0flows
> +ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +
> +# Set the option "centralize_routing=true" for lr0-public.
> +check ovn-nbctl --wait=sb set logical_router_port lr0-public options:centralize_routing=true
> +
> +# Check that the lflows are as expected and there is cr port created for public-lr0.
> +check_flows_cr_port_for_public_lr0
> +
> +# Set the type of ln-public back to localnet
> +check ovn-nbctl --wait=sb lsp-set-type ln-public localnet
> +
> +# Check that the lflows are as expected when public has localnet port.
> +check_flows_no_cr_port_for_public_lr0
> +
> +# Delete the localnet port
> +check ovn-nbctl --wait=sb lsp-del ln-public
> +
> +# Check that the lflows are as expected when public has no localnet port.
> +check_flows_cr_port_for_public_lr0
> +
> +# Create multiple gateway ports.  chassisresident port should not be
> +# created for 'public-lr0' even if there is no localnet port on 'public'
> +# logical switch.
> +check ovn-nbctl --wait=sb lrp-set-gateway-chassis lr0-sw0 gw1
> +# check that there is no port binding cr-public-lr0
> +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +ovn-sbctl dump-flows lr0 > lr0flows
> +ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +AT_CLEANUP
> +])
> diff --git a/tests/ovn.at b/tests/ovn.at
> index 582ec56485..c468673f0c 100644
> --- a/tests/ovn.at
> +++ b/tests/ovn.at
> @@ -21227,10 +21227,10 @@ ovn-nbctl lsp-add sw0 rp-sw0 -- set Logical_Switch_Port rp-sw0 \
>       type=router options:router-port=sw0 \
>       -- lsp-set-addresses rp-sw0 router
>   
> -ovn-nbctl lrp-add lr0 sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
> -    -- lrp-set-gateway-chassis sw1 hv2
> +ovn-nbctl lrp-add lr0 lr0-sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
> +    -- lrp-set-gateway-chassis lr0-sw1 hv2
>   ovn-nbctl lsp-add sw1 rp-sw1 -- set Logical_Switch_Port rp-sw1 \
> -    type=router options:router-port=sw1 \
> +    type=router options:router-port=lr0-sw1 \
>       -- lsp-set-addresses rp-sw1 router
>   
>   ovn-nbctl lsp-add sw0 sw0-p0 \
> @@ -21242,6 +21242,8 @@ ovn-nbctl lsp-add sw0 sw0-p1 \
>   ovn-nbctl lsp-add sw1 sw1-p0 \
>       -- lsp-set-addresses sw1-p0 unknown
>   
> +check ovn-nbctl lsp-add sw1 ln-sw1 -- lsp-set-type ln-sw1 localnet
> +
>   ovn-nbctl lr-nat-add lr0 snat 172.16.1.1 192.168.1.0/24
>   ovn-nbctl lr-nat-add lr0 snat 2002::1 2001::/64
>
Felix Huettner Aug. 5, 2024, 12:44 p.m. UTC | #2 
On Mon, Jul 29, 2024 at 10:38:49PM -0400, numans@ovn.org wrote:
> From: Numan Siddique <numans@ovn.org>
> 
> Consider a deployment with the below logical resources:
> 
> 1. A bridged logical switch 'public' with a port - P1 and a localnet
>    port ln-public.
> 2. A logical router 'R'
> 3. Logical switch 'public' connected to R via logical switch/router port
>    peers (public-R and R-public).
> 4. R-public is distributed gateway port with its network as 172.16.0.0/24
> 5. NATs (dnat_and_snat) configured in 'R'.
> 6. And a few overlay logical switches S1, S2 to R.
> 
> Any traffic from logical port - P1 of public logical switch destined to
> S1 or S2's logical ports goes out of the source chassis
> (where P1 resides) via the localnet port and reaches the gateway chassis
> which handles the routing.
> 
> There are couple of traffic flow scenarios which doesn't work if the
> logical switch 'public' doesn't have a localnet port.
> 
> 1. Traffic from port - P1 destined to logical switches S1 or S2 gets
>    dropped in the source chassis.  The packet enters the router R's
>    pipeline, but it gets dropped in the 'lr_in_admission' stage since
>    the logical flow to allow traffic destined to the distributed gateway
>    port MAC is installed only on the gateway chassis.
> 
> 2. NAT doesn't work as expected.
> 
> In order to suppose this use case (of a logical switch not having a
> localnet port, but has a distributed gateway port and NATs), this patch
> supports the option 'centralize_routing', which can be configured on
> the distributed gateway port (R-public in the example above).
> If this option is set, then routing is centralized on the gateway
> chassis for the traffic destined to the R-public's networks
> (172.16.0.0/24 for the above example).  Traffic from P1 will be
> tunnelled to the gateway chassis.
> 
> ovn-northd creates a chassisresident port (cr-public-R) for the
> logical switch port - public-R, along with cr-R-public inorder to
> centralize the traffic.
> 
> This feature gets enabled for the distributed gateway port R-public if
>   - The above option is set to true in the R-public's options column.
>   - The logical switch 'public' doesn't have any localnet ports.
>   - And R-public is the only distributed gateway port of R.
> 
> Distributed NAT (i.e if external_mac and router_port is set) is
> not supported and instead the router port mac is used for such traffic
> and centralized on the gateway chassis.
> 
> Reported-at: https://issues.redhat.com/browse/FDP-364
> Acked-by: Mark Michelson <mmichels@redhat.com>
> Signed-off-by: Numan Siddique <numans@ovn.org>
> ---
> 
> v1 -> v2
> -------
>    * Corrected the NEWS item entry for the new option.
>    * Rebased and resolved conflicts.
> 
> Note: This patch is the continuation from this series - https://patchwork.ozlabs.org/project/ovn/patch/20240606214432.168750-1-numans@ovn.org/
>       Resetted the version number since the new option changed from LSP
>       to LRP.
> 
> 
>  NEWS                      |   3 +
>  controller/physical.c     |   4 +
>  northd/northd.c           | 257 +++++++++++++++----
>  northd/northd.h           |   1 +
>  ovn-nb.xml                |  34 +++
>  tests/multinode-macros.at |   2 +-
>  tests/multinode.at        | 177 +++++++++++++
>  tests/ovn-northd.at       | 516 +++++++++++++++++++++++++++++++++++++-
>  tests/ovn.at              |   8 +-
>  9 files changed, 952 insertions(+), 50 deletions(-)
> 
> diff --git a/NEWS b/NEWS
> index 87e326f21e..8440a74677 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -47,6 +47,9 @@ Post v24.03.0
>    - Add support for CT zone limit that can be specified per LR
>      (options:ct-zone-limit), LS (other_config:ct-zone-limit) or LSP
>      (options:ct-zone-limit).
> +  - A new LRP option 'centralize_routing' has been added to a
> +    distributed gateway port to centralize routing if the logical
> +    switch of its peer doesn't have a localnet port.
>  
>  OVN v24.03.0 - 01 Mar 2024
>  --------------------------
> diff --git a/controller/physical.c b/controller/physical.c
> index 3c0200c383..9e04ad5f22 100644
> --- a/controller/physical.c
> +++ b/controller/physical.c
> @@ -1610,6 +1610,10 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name,
>                                                      ct_zones);
>              put_zones_ofpacts(&zone_ids, ofpacts_p);
>  
> +            /* Clear the MFF_INPORT.  Its possible that the same packet may
> +             * go out from the same tunnel inport. */
> +            put_load(ofp_to_u16(OFPP_NONE), MFF_IN_PORT, 0, 16, ofpacts_p);
> +
>              /* Resubmit to table 41. */
>              put_resubmit(OFTABLE_CHECK_LOOPBACK, ofpacts_p);
>          }
> diff --git a/northd/northd.c b/northd/northd.c
> index 5c2fd74ff1..4f59d4f1a3 100644
> --- a/northd/northd.c
> +++ b/northd/northd.c
> @@ -2107,6 +2107,55 @@ parse_lsp_addrs(struct ovn_port *op)
>      }
>  }
>  
> +static struct ovn_port *
> +create_cr_port(struct ovn_port *op, struct hmap *ports,
> +               struct ovs_list *both_dbs, struct ovs_list *nb_only)
> +{
> +    char *redirect_name = ovn_chassis_redirect_name(
> +        op->nbsp ? op->nbsp->name : op->nbrp->name);
> +
> +    struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> +    if (crp && crp->sb && crp->sb->datapath == op->od->sb) {
> +        ovn_port_set_nb(crp, NULL, op->nbrp);

Hi Numan,

i think you need to pass op->nbsp instead of NULL here.

Otherwise i observe the following crash:

northd/northd.c:4371:48: runtime error: member access within null pointer of type 'const struct nbrec_logical_router_port'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior northd/northd.c:4371:48
AddressSanitizer:DEADLYSIGNAL
=================================================================
==84927==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000180 (pc 0x57ab3854f04a bp 0x7ffc54d1ba30 sp 0x7ffc54d1ba20 T0)
==84927==The signal is caused by a READ memory access.
==84927==Hint: address points to the zero page.
    #0 0x57ab3854f04a in hmap_first_with_hash /home/ovn/ovs/./include/openvswitch/hmap.h:360:38
    #1 0x57ab3854fedf in smap_find__ /home/ovn/ovs/lib/smap.c:421:5
    #2 0x57ab3854f7b8 in smap_get_node /home/ovn/ovs/lib/smap.c:217:12
    #3 0x57ab3854f74f in smap_get_def /home/ovn/ovs/lib/smap.c:208:30
    #4 0x57ab3854f726 in smap_get /home/ovn/ovs/lib/smap.c:200:12
    #5 0x57ab3854f862 in smap_get_int /home/ovn/ovs/lib/smap.c:240:25
    #6 0x57ab383222eb in ovn_port_assign_requested_tnl_id /home/ovn/northd/northd.c:4372:27
    #7 0x57ab383072fa in build_ports /home/ovn/northd/northd.c:4454:9
    #8 0x57ab38301457 in ovnnb_db_run /home/ovn/northd/northd.c:18023:5
    #9 0x57ab3841d99e in en_northd_run /home/ovn/northd/en-northd.c:137:5
    #10 0x57ab384599b2 in engine_recompute /home/ovn/lib/inc-proc-eng.c:411:5
    #11 0x57ab38459d6e in engine_run_node /home/ovn/lib/inc-proc-eng.c:473:9
    #12 0x57ab38459ec3 in engine_run /home/ovn/lib/inc-proc-eng.c:524:9
    #13 0x57ab38430c5d in inc_proc_northd_run /home/ovn/northd/inc-proc-northd.c:420:5
    #14 0x57ab3841bb2f in main /home/ovn/northd/ovn-northd.c:970:32
    #15 0x7ab516c2a1c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #16 0x7ab516c2a28a in __libc_start_main csu/../csu/libc-start.c:360:3
    #17 0x57ab3818cda4 in _start (/usr/local/bin/ovn-northd+0x126da4) (BuildId: 9163c1ae41fe5e3ca08fb7f1741bbea50426b335)

I guess this is because nbrp will be null as the port is derived from a lsp and
therefor we later would have neither nbrp nor nbsp.

In case the port does not yet exist in the ports hmap it is below
generated correctly with nbrp and nbsp whatever is set.

Thanks
Felix

> +        ovs_list_remove(&crp->list);
> +        ovs_list_push_back(both_dbs, &crp->list);
> +    } else {
> +        crp = ovn_port_create(ports, redirect_name,
> +                              op->nbsp, op->nbrp, NULL);
> +        ovs_list_push_back(nb_only, &crp->list);
> +    }
> +
> +    crp->primary_port = op;
> +    op->cr_port = crp;
> +    crp->od = op->od;
> +    free(redirect_name);
> +
> +    return crp;
> +}
> +
> +/* Returns true if chassis resident port needs to be created for
> + * op's peer logical switch.  False otherwise.
> + *
> + * Chassis resident port needs to be created if the following
> + * conditionsd are met:
> + *   - op is a distributed gateway port
> + *   - op has the option 'centralize_routing' set to true
> + *   - op is the only distributed gateway port attached to its
> + *     router
> + *   - op's peer logical switch has no localnet ports.
> + */
> +static bool
> +peer_needs_cr_port_creation(struct ovn_port *op)
> +{
> +    if ((op->nbrp->n_gateway_chassis || op->nbrp->ha_chassis_group)
> +        && op->od->n_l3dgw_ports == 1 && op->peer && op->peer->nbsp
> +        && !op->peer->od->n_localnet_ports) {
> +        return smap_get_bool(&op->nbrp->options, "centralize_routing", false);
> +    }
> +
> +    return false;
> +}
> +
>  static void
>  join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                     struct hmap *ls_datapaths, struct hmap *lr_datapaths,
> @@ -2214,9 +2263,10 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>              tag_alloc_add_existing_tags(tag_alloc_table, nbsp);
>          }
>      }
> +
> +    struct hmapx dgps = HMAPX_INITIALIZER(&dgps);
>      HMAP_FOR_EACH (od, key_node, lr_datapaths) {
>          ovs_assert(od->nbr);
> -        size_t n_allocated_l3dgw_ports = 0;
>          for (size_t i = 0; i < od->nbr->n_ports; i++) {
>              const struct nbrec_logical_router_port *nbrp
>                  = od->nbr->ports[i];
> @@ -2280,10 +2330,7 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                      redirect_type && !strcasecmp(redirect_type, "bridged");
>              }
>  
> -            if (op->nbrp->ha_chassis_group ||
> -                op->nbrp->n_gateway_chassis) {
> -                /* Additional "derived" ovn_port crp represents the
> -                 * instance of op on the gateway chassis. */
> +            if (op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis) {
>                  const char *gw_chassis = smap_get(&op->od->nbr->options,
>                                                 "chassis");
>                  if (gw_chassis) {
> @@ -2292,34 +2339,9 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                      VLOG_WARN_RL(&rl, "Bad configuration: distributed "
>                                   "gateway port configured on port %s "
>                                   "on L3 gateway router", nbrp->name);
> -                    continue;
> -                }
> -
> -                char *redirect_name =
> -                    ovn_chassis_redirect_name(nbrp->name);
> -                struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> -                if (crp && crp->sb && crp->sb->datapath == od->sb) {
> -                    ovn_port_set_nb(crp, NULL, nbrp);
> -                    ovs_list_remove(&crp->list);
> -                    ovs_list_push_back(both, &crp->list);
>                  } else {
> -                    crp = ovn_port_create(ports, redirect_name,
> -                                          NULL, nbrp, NULL);
> -                    ovs_list_push_back(nb_only, &crp->list);
> -                }
> -                crp->primary_port = op;
> -                op->cr_port = crp;
> -                crp->od = od;
> -                free(redirect_name);
> -
> -                /* Add to l3dgw_ports in od, for later use during flow
> -                 * creation. */
> -                if (od->n_l3dgw_ports == n_allocated_l3dgw_ports) {
> -                    od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> -                                                 &n_allocated_l3dgw_ports,
> -                                                 sizeof *od->l3dgw_ports);
> +                    hmapx_add(&dgps, op);
>                  }
> -                od->l3dgw_ports[od->n_l3dgw_ports++] = op;
>             }
>          }
>      }
> @@ -2376,12 +2398,6 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                          arp_proxy, op->nbsp->name);
>                  }
>              }
> -
> -            /* Only used for the router type LSP whose peer is l3dgw_port */
> -            if (op->peer && is_l3dgw_port(op->peer)) {
> -                op->enable_router_port_acl = smap_get_bool(
> -                    &op->nbsp->options, "enable_router_port_acl", false);
> -            }
>          } else if (op->nbrp && op->nbrp->peer && !is_cr_port(op)) {
>              struct ovn_port *peer = ovn_port_find(ports, op->nbrp->peer);
>              if (peer) {
> @@ -2402,6 +2418,57 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>          }
>      }
>  
> +    struct hmapx_node *hmapx_node;
> +    HMAPX_FOR_EACH (hmapx_node, &dgps) {
> +        op = hmapx_node->data;
> +        od = op->od;
> +        ovs_assert(op->nbrp);
> +        ovs_assert(op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis);
> +
> +        /* Additional "derived" ovn_port crp represents the instance of op on
> +         * the gateway chassis. */
> +        struct ovn_port *crp = create_cr_port(op, ports, both, nb_only);
> +        ovs_assert(crp);
> +
> +        /* Add to l3dgw_ports in od, for later use during flow creation. */
> +        if (od->n_l3dgw_ports == od->n_allocated_l3dgw_ports) {
> +            od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> +                                        &od->n_allocated_l3dgw_ports,
> +                                        sizeof *od->l3dgw_ports);
> +        }
> +        od->l3dgw_ports[od->n_l3dgw_ports++] = op;
> +
> +        if (op->peer && op->peer->nbsp) {
> +            /* Only used for the router type LSP whose peer is l3dgw_port */
> +            op->peer->enable_router_port_acl = smap_get_bool(
> +                    &op->peer->nbsp->options, "enable_router_port_acl", false);
> +        }
> +    }
> +
> +
> +    /* Create chassisresident port for the distributed gateway port's (DGP)
> +     * peer if
> +     *  - DGP's router has only one DGP and
> +     *  - Its peer is a logical switch port and
> +     *  - It's peer's logical switch has no localnet ports and
> +     *  - option 'centralize_routing' is set to true for the DGP.
> +     *
> +     * This is required to support
> +     *   - NAT via geneve (for the overlay provider networks) and
> +     *   - to centralize routing on the gateway chassis for the traffic
> +     *     destined to the DGP's networks.
> +     *
> +     * Future enhancement: Support 'centralizerouting' for all the DGP's
> +     * of a logical router.
> +     * */
> +    HMAPX_FOR_EACH (hmapx_node, &dgps) {
> +        op = hmapx_node->data;
> +        if (peer_needs_cr_port_creation(op)) {
> +            create_cr_port(op->peer, ports, both, nb_only);
> +        }
> +    }
> +    hmapx_destroy(&dgps);
> +
>      /* Wait until all ports have been connected to add to IPAM since
>       * it relies on proper peers to be set
>       */
> @@ -3184,16 +3251,28 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn,
>               * type "l3gateway". */
>              if (chassis) {
>                  sbrec_port_binding_set_type(op->sb, "l3gateway");
> +            } else if (is_cr_port(op)) {
> +                sbrec_port_binding_set_type(op->sb, "chassisredirect");
> +                ovs_assert(op->primary_port->peer);
> +                ovs_assert(op->primary_port->peer->cr_port);
> +                ovs_assert(op->primary_port->peer->cr_port->sb);
> +                sbrec_port_binding_set_ha_chassis_group(
> +                    op->sb,
> +                    op->primary_port->peer->cr_port->sb->ha_chassis_group);
> +
>              } else {
>                  sbrec_port_binding_set_type(op->sb, "patch");
>              }
>  
>              const char *router_port = smap_get(&op->nbsp->options,
>                                                 "router-port");
> -            if (router_port || chassis) {
> +            if (router_port || chassis || is_cr_port(op)) {
>                  struct smap new;
>                  smap_init(&new);
> -                if (router_port) {
> +
> +                if (is_cr_port(op)) {
> +                    smap_add(&new, "distributed-port", op->nbsp->name);
> +                } else if (router_port) {
>                      smap_add(&new, "peer", router_port);
>                  }
>                  if (chassis) {
> @@ -8155,9 +8234,27 @@ build_lswitch_rport_arp_req_flow(
>      struct lflow_ref *lflow_ref)
>  {
>      struct ds match   = DS_EMPTY_INITIALIZER;
> +    struct ds m       = DS_EMPTY_INITIALIZER;
>      struct ds actions = DS_EMPTY_INITIALIZER;
>  
> -    arp_nd_ns_match(ips, addr_family, &match);
> +    arp_nd_ns_match(ips, addr_family, &m);
> +    ds_clone(&match, &m);
> +
> +    bool has_cr_port = patch_op->cr_port;
> +
> +    /* If the patch_op has a chassis resident port, it means
> +     *    - its peer is a distributed gateway port (DGP) and
> +     *    - routing is centralized for the DGP's networks on
> +     *      the configured gateway chassis.
> +     *
> +     * If that's the case, make sure that the packets destined to
> +     * the DGP's MAC are sent to the chassis where the DGP resides.
> +     * */
> +
> +    if (has_cr_port) {
> +        ds_put_format(&match, " && is_chassis_resident(%s)",
> +                      patch_op->cr_port->json_key);
> +    }
>  
>      /* Send a the packet to the router pipeline.  If the switch has non-router
>       * ports then flood it there as well.
> @@ -8179,6 +8276,31 @@ build_lswitch_rport_arp_req_flow(
>                                  lflow_ref);
>      }
>  
> +    if (has_cr_port) {
> +        ds_clear(&match);
> +        ds_put_format(&match, "%s && !is_chassis_resident(%s)", ds_cstr(&m),
> +                      patch_op->cr_port->json_key);
> +        ds_clear(&actions);
> +        if (od->n_router_ports != od->nbs->n_ports) {
> +            ds_put_format(&actions, "clone {outport = %s; output; }; "
> +                                    "outport = \""MC_FLOOD_L2"\"; output;",
> +                          patch_op->cr_port->json_key);
> +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> +                                    priority, ds_cstr(&match),
> +                                    ds_cstr(&actions), stage_hint,
> +                                    lflow_ref);
> +        } else {
> +            ds_put_format(&actions, "outport = %s; output;",
> +                          patch_op->cr_port->json_key);
> +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> +                                    priority, ds_cstr(&match),
> +                                    ds_cstr(&actions),
> +                                    stage_hint,
> +                                    lflow_ref);
> +        }
> +    }
> +
> +    ds_destroy(&m);
>      ds_destroy(&match);
>      ds_destroy(&actions);
>  }
> @@ -9548,7 +9670,11 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
>                                  struct ds *actions, struct ds *match)
>  {
>      ovs_assert(op->nbsp);
> -    if (lsp_is_external(op->nbsp)) {
> +
> +    /* Note: A switch port can also have a chassis resident derived port.
> +     * Check if 'op' is a chassis resident dervied port. If so, skip
> +     * adding unicast lookup flows for this port. */
> +    if (lsp_is_external(op->nbsp) || is_cr_port(op)) {
>          return;
>      }
>  
> @@ -9566,8 +9692,6 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
>                             "outport = \""MC_UNKNOWN "\"; output;"
>                           : "outport = %s; output;")
>                           : debug_drop_action();
> -    ds_clear(actions);
> -    ds_put_format(actions, action, op->json_key);
>  
>      if (lsp_is_router(op->nbsp) && op->peer && op->peer->nbrp) {
>          /* For ports connected to logical routers add flows to bypass the
> @@ -9614,14 +9738,43 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
>              if (add_chassis_resident_check) {
>                  ds_put_format(match, " && is_chassis_resident(%s)", json_key);
>              }
> +        } else if (op->cr_port) {
> +            /* If the op has a chassis resident port, it means
> +             *   - its peer is a distributed gateway port (DGP) and
> +             *   - routing is centralized for the DGP's networks on
> +             *     the configured gateway chassis.
> +             *
> +             * If that's the case, make sure that the packets destined to
> +             * the DGP's MAC are sent to the chassis where the DGP resides.
> +             * */
> +            ds_clear(actions);
> +            ds_put_format(actions, action, op->cr_port->json_key);
> +
> +            struct ds m = DS_EMPTY_INITIALIZER;
> +            ds_put_format(&m, "eth.dst == %s && !is_chassis_resident(%s)",
> +                          op->peer->lrp_networks.ea_s,
> +                          op->cr_port->json_key);
> +
> +            ovn_lflow_add_with_hint(lflows, op->od,
> +                                    S_SWITCH_IN_L2_LKUP, 50,
> +                                    ds_cstr(&m), ds_cstr(actions),
> +                                    &op->nbsp->header_,
> +                                    op->lflow_ref);
> +            ds_destroy(&m);
> +            ds_put_format(match, " && is_chassis_resident(%s)",
> +                          op->cr_port->json_key);
>          }
>  
> +        ds_clear(actions);
> +        ds_put_format(actions, action, op->json_key);
>          ovn_lflow_add_with_hint(lflows, op->od,
>                                  S_SWITCH_IN_L2_LKUP, 50,
>                                  ds_cstr(match), ds_cstr(actions),
>                                  &op->nbsp->header_,
>                                  op->lflow_ref);
>      } else {
> +        ds_clear(actions);
> +        ds_put_format(actions, action, op->json_key);
>          for (size_t i = 0; i < op->n_lsp_addrs; i++) {
>              ds_clear(match);
>              ds_put_format(match, "eth.dst == %s", op->lsp_addrs[i].ea_s);
> @@ -11725,6 +11878,14 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op,
>          return;
>      }
>  
> +    if (op->peer && op->peer->cr_port) {
> +        /* We don't add the below flows if the router port's peer has
> +         * a chassisresident port.  That's because routing is centralized on
> +         * the gateway chassis for the router port networks/subnets.
> +         */
> +        return;
> +    }
> +
>      /* Mac address to use when replying to ARP/NS. */
>      const char *mac_s = REG_INPORT_ETH_ADDR;
>      struct eth_addr mac;
> @@ -15109,6 +15270,16 @@ lrouter_check_nat_entry(const struct ovn_datapath *od,
>      /* For distributed router NAT, determine whether this NAT rule
>       * satisfies the conditions for distributed NAT processing. */
>      *distributed = false;
> +
> +    /* NAT cannnot be distributed if the DGP's peer
> +     * has a chassisresident port (as the routing is centralized
> +     * on the gateway chassis for the DGP's networks/subnets.)
> +     */
> +    struct ovn_port *l3dgw_port = *nat_l3dgw_port;
> +    if (l3dgw_port && l3dgw_port->peer && l3dgw_port->peer->cr_port) {
> +        return 0;
> +    }
> +
>      if (od->n_l3dgw_ports && !strcmp(nat->type, "dnat_and_snat") &&
>          nat->logical_port && nat->external_mac) {
>          if (eth_addr_from_string(nat->external_mac, mac)) {
> diff --git a/northd/northd.h b/northd/northd.h
> index d4a8d75abc..d7c9655916 100644
> --- a/northd/northd.h
> +++ b/northd/northd.h
> @@ -325,6 +325,7 @@ struct ovn_datapath {
>       * will be NULL. */
>      struct ovn_port **l3dgw_ports;
>      size_t n_l3dgw_ports;
> +    size_t n_allocated_l3dgw_ports;
>  
>      /* router datapath has a logical port with redirect-type set to bridged. */
>      bool redirect_bridged;
> diff --git a/ovn-nb.xml b/ovn-nb.xml
> index a4362a4ef1..217d1cd1fe 100644
> --- a/ovn-nb.xml
> +++ b/ovn-nb.xml
> @@ -3499,6 +3499,40 @@ or
>            <ref column="options" key="gateway_mtu"/> option.
>          </p>
>        </column>
> +
> +      <column name="options" key="centralize_routing"
> +              type='{"type": "boolean"}'>
> +        <p>
> +          This option is applicable only if the router port is a
> +          distributed gateway port i.e if the <ref table="Logical_Router_Port"
> +          column="ha_chassis_group"/> column or
> +          <ref table="Logical_Router_Port" column="gateway_chassis"/>
> +          is set.
> +        </p>
> +
> +        <p>
> +          If set to <code>true</code>, routing for the router port's
> +          networks (set in the column <ref table="Logical_Router_Port"
> +          column="networks"/>) is centralized on the gateway chassis
> +          which claims this distributed gateway port.
> +        </p>
> +
> +        <p>
> +          Additionally for this option to take effect, below conditions
> +          must be met:
> +        </p>
> +
> +        <ul>
> +          <li>
> +            The Logical router has only one distributed gateway port.
> +          </li>
> +
> +          <li>
> +            The router port's peer logical switch has no localnet ports.
> +          </li>
> +
> +        </ul>
> +      </column>
>      </group>
>  
>      <group title="Attachment">
> diff --git a/tests/multinode-macros.at b/tests/multinode-macros.at
> index 786e564860..757917626c 100644
> --- a/tests/multinode-macros.at
> +++ b/tests/multinode-macros.at
> @@ -92,7 +92,7 @@ m_count_rows() {
>  m_check_row_count() {
>      local db=$(parse_db $1) table=$(parse_table $1); shift
>      local count=$1; shift
> -    local found=$(m_count_rows $c $db:$table "$@")
> +    local found=$(m_count_rows $db:$table "$@")
>      echo
>      echo "Checking for $count rows in $db $table${1+ with $*}... found $found"
>      if test "$count" != "$found"; then
> diff --git a/tests/multinode.at b/tests/multinode.at
> index a7231130ac..a725414165 100644
> --- a/tests/multinode.at
> +++ b/tests/multinode.at
> @@ -1033,6 +1033,183 @@ M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [sh -c 'dd bs=512 count=2 if=/dev/uran
>  done
>  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ip route get 10.0.0.1 dev sw0p1 | grep -q 'mtu 942'])
>  
> +# Reset back to geneve tunnels
> +for c in ovn-chassis-1 ovn-chassis-2 ovn-gw-1
> +do
> +    m_as $c ovs-vsctl set open . external-ids:ovn-encap-type=geneve
> +done
> +
> +AT_CLEANUP
> +
> +AT_SETUP([ovn multinode NAT on a provider network with no localnet ports])
> +
> +# Check that ovn-fake-multinode setup is up and running
> +check_fake_multinode_setup
> +
> +# Delete the multinode NB and OVS resources before starting the test.
> +cleanup_multinode_resources
> +
> +check multinode_nbctl ls-add sw0
> +check multinode_nbctl lsp-add sw0 sw0-port1
> +check multinode_nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03 10.0.0.3 1000::3"
> +check multinode_nbctl lsp-add sw0 sw0-port2
> +check multinode_nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:04 10.0.0.4 1000::4"
> +
> +m_as ovn-chassis-1 /data/create_fake_vm.sh sw0-port1 sw0p1 50:54:00:00:00:03 10.0.0.3 24 10.0.0.1 1000::3/64 1000::a
> +m_as ovn-chassis-2 /data/create_fake_vm.sh sw0-port2 sw0p2 50:54:00:00:00:04 10.0.0.4 24 10.0.0.1 1000::4/64 1000::a
> +
> +m_wait_for_ports_up
> +
> +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.4 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +# Create the second logical switch with one port
> +check multinode_nbctl ls-add sw1
> +check multinode_nbctl lsp-add sw1 sw1-port1
> +check multinode_nbctl lsp-set-addresses sw1-port1 "40:54:00:00:00:03 20.0.0.3 2000::3"
> +
> +# Create a logical router and attach both logical switches
> +check multinode_nbctl lr-add lr0
> +check multinode_nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 1000::a/64
> +check multinode_nbctl lsp-add sw0 sw0-lr0
> +check multinode_nbctl lsp-set-type sw0-lr0 router
> +check multinode_nbctl lsp-set-addresses sw0-lr0 router
> +check multinode_nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> +
> +check multinode_nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:02 20.0.0.1/24 2000::a/64
> +check multinode_nbctl lsp-add sw1 sw1-lr0
> +check multinode_nbctl lsp-set-type sw1-lr0 router
> +check multinode_nbctl lsp-set-addresses sw1-lr0 router
> +check multinode_nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> +
> +m_as ovn-chassis-2 /data/create_fake_vm.sh sw1-port1 sw1p1 40:54:00:00:00:03 20.0.0.3 24 20.0.0.1 2000::3/64 2000::a
> +
> +# create exteranl connection for N/S traffic
> +check multinode_nbctl ls-add public
> +check multinode_nbctl lsp-add public ln-public
> +check multinode_nbctl lsp-set-type ln-public localnet
> +check multinode_nbctl lsp-set-addresses ln-public unknown
> +check multinode_nbctl lsp-set-options ln-public network_name=public
> +
> +check multinode_nbctl lrp-add lr0 lr0-public 00:11:22:00:ff:01 172.20.0.100/24
> +check multinode_nbctl lsp-add public public-lr0
> +check multinode_nbctl lsp-set-type public-lr0 router
> +check multinode_nbctl lsp-set-addresses public-lr0 router
> +check multinode_nbctl lsp-set-options public-lr0 router-port=lr0-public
> +check multinode_nbctl lrp-set-gateway-chassis lr0-public ovn-gw-1 10
> +
> +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
> +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.120 20.0.0.3
> +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 10.0.0.0/24
> +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 20.0.0.0/24
> +
> +# Create a logical port pub-p1 and bind it in ovn-chassis-1
> +check multinode_nbctl lsp-add public public-port1
> +check multinode_nbctl lsp-set-addresses public-port1 "60:54:00:00:00:03 172.168.0.50"
> +
> +m_as ovn-chassis-1 /data/create_fake_vm.sh public-port1 pubp1 60:54:00:00:00:03 172.20.0.50 24 172.20.0.100
> +
> +check multinode_nbctl --wait=hv sync
> +
> +# First do basic ping tests before deleting the localnet port - ln-public.
> +# Once the localnet port is deleted from public ls, routing for 172.20.0.0/24
> +# is centralized on ovn-gw-1.
> +
> +# This function checks the North-South traffic.
> +run_ns_traffic() {
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.110], [ignore], [ignore])
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.120], [ignore], [ignore])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-2], [sw1p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  # Now ping from pubp1 to 172.20.0.100, 172.20.0.110, 172.20.0.120, 10.0.0.3 and 20.0.0.3
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.3 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 20.0.0.3 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +}
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
> +# Delete the localnet port by changing the type of ln-public to VIF port.
> +check multinode_nbctl --wait=hv lsp-set-type ln-public ""
> +
> +# cr-port should not be created for public-lr0 since the option
> +# centralize_routing=true is not yet set for lr0-public.
> +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +# Set the option - centralize_routing now.
> +check multinode_nbctl --wait=hv set logical_router_port lr0-public options:centralize_routing=true
> +
> +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> +m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
> +# Re-add the localnet port
> +check multinode_nbctl --wait=hv lsp-set-type ln-public localnet
> +
> +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
> +# Delete the ln-public port this time.
> +check multinode_nbctl --wait=hv lsp-del ln-public
> +
> +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> +m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
>  AT_CLEANUP
>  
>  AT_SETUP([ovn provider network - always_tunnel])
> diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
> index 57f89a7746..649c20f285 100644
> --- a/tests/ovn-northd.at
> +++ b/tests/ovn-northd.at
> @@ -2187,7 +2187,7 @@ match=(inport == "lrp-public" && arp.op == 1 && arp.tpa == 43.43.43.4 && is_chas
>  action=(eth.dst = eth.src; eth.src = 00:00:00:00:00:02; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 00:00:00:00:00:02; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
>  ])
>  
> -# xreg0[0..47] isn't used anywhere else.
> +# xreg0[[0..47]] isn't used anywhere else.
>  AT_CHECK([ovn-sbctl lflow-list | grep "xreg0\[[0..47\]]" | grep -vE 'lr_in_admission|lr_in_ip_input'], [1], [])
>  
>  AT_CLEANUP
> @@ -5503,13 +5503,14 @@ AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | grep "192.168.4.100" | grep "_MC_flo
>  
>  AS_BOX([Configuring ro1-ls1 router port as a gateway router port])
>  
> -ovn-nbctl --wait=sb lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> +check ovn-nbctl  lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> +check ovn-nbctl --wait=sb lsp-add ls1 ln-ls1 -- lsp-set-type ln-ls1 localnet
>  
>  ovn-sbctl lflow-list ls1 > ls1_lflows
>  AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | ovn_strip_lflows], [0], [dnl
>    table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
>    table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> -  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01), action=(outport = "ls1-ro1"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01 && is_chassis_resident("cr-ro1-ls1")), action=(outport = "ls1-ro1"; output;)
>    table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:02), action=(outport = "vm1"; output;)
>    table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
>    table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:01:01} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> @@ -12475,3 +12476,512 @@ AT_CHECK([ovn-sbctl dump-flows lr | grep lr_in_dnat | ovn_strip_lflows], [0], [d
>  
>  AT_CLEANUP
>  ])
> +
> +OVN_FOR_EACH_NORTHD_NO_HV([
> +AT_SETUP([NAT on a provider network with no localnet ports])
> +AT_KEYWORDS([NAT])
> +ovn_start
> +
> +check ovn-nbctl -- ls-add sw0 -- ls-add sw1
> +check ovn-nbctl lsp-add sw0 sw0-port1
> +check ovn-nbctl lr-add lr0
> +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
> +check ovn-nbctl lsp-add sw0 sw0-lr0
> +check ovn-nbctl lsp-set-type sw0-lr0 router
> +check ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01
> +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> +
> +check ovn-nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:03 20.0.0.1/24
> +check ovn-nbctl lsp-add sw1 sw1-lr0
> +check ovn-nbctl lsp-set-type sw1-lr0 router
> +check ovn-nbctl lsp-set-addresses sw1-lr0 router
> +check ovn-nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> +
> +check ovn-sbctl chassis-add gw1 geneve 127.0.0.1
> +check ovn-nbctl ls-add public
> +check ovn-nbctl lsp-add public pub-p1
> +
> +# localnet port
> +check ovn-nbctl lsp-add public ln-public
> +check ovn-nbctl lsp-set-type ln-public localnet
> +check ovn-nbctl lsp-set-addresses ln-public unknown
> +check ovn-nbctl lsp-set-options ln-public network_name=public
> +
> +check ovn-nbctl lrp-add lr0 lr0-public 00:00:00:00:ff:02 172.168.0.10/24
> +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1
> +
> +check ovn-nbctl lsp-add public public-lr0
> +check ovn-nbctl lsp-set-type public-lr0 router
> +check ovn-nbctl lsp-set-addresses public-lr0 router
> +check ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public
> +
> +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
> +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.120 20.0.0.3
> +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 10.0.0.0/24
> +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 20.0.0.0/24
> +
> +check ovn-nbctl --wait=sb sync
> +
> +check_flows_no_cr_port_for_public_lr0() {
> +  # check that there is no port binding cr-public-lr0
> +  check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +  ovn-sbctl dump-flows lr0 > lr0flows
> +  ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
> +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +])
> +
> +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +])
> +
> +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +])
> +
> +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +])
> +
> +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> +])
> +
> +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +])
> +
> +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +])
> +
> +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-lr0-public")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +}
> +
> +check_flows_cr_port_for_public_lr0() {
> +  # check that there is port binding cr-public-lr0
> +  check_row_count Port_Binding 1 logical_port=cr-public-lr0
> +  check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> +
> +  ovn-sbctl dump-flows lr0 > lr0flows
> +  ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
> +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +])
> +
> +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +])
> +
> +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +])
> +
> +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +])
> +
> +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> +])
> +
> +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +])
> +
> +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +])
> +
> +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-public-lr0")), action=(outport = "cr-public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-public-lr0")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +}
> +
> +# Check that the lflows are as expected when public has localnet port.
> +check_flows_no_cr_port_for_public_lr0
> +
> +# Remove the localnet port from public logical switch.
> +check ovn-nbctl --wait=sb lsp-set-type ln-public ""
> +
> +# Check that the lflows are as expected and there is no cr port
> +# created for "public-lr0"  when public has no localnet port
> +# since public doesn't have the option "overlay_provider_network=true"
> +# set.
> +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +ovn-sbctl dump-flows lr0 > lr0flows
> +ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +
> +# Set the option "centralize_routing=true" for lr0-public.
> +check ovn-nbctl --wait=sb set logical_router_port lr0-public options:centralize_routing=true
> +
> +# Check that the lflows are as expected and there is cr port created for public-lr0.
> +check_flows_cr_port_for_public_lr0
> +
> +# Set the type of ln-public back to localnet
> +check ovn-nbctl --wait=sb lsp-set-type ln-public localnet
> +
> +# Check that the lflows are as expected when public has localnet port.
> +check_flows_no_cr_port_for_public_lr0
> +
> +# Delete the localnet port
> +check ovn-nbctl --wait=sb lsp-del ln-public
> +
> +# Check that the lflows are as expected when public has no localnet port.
> +check_flows_cr_port_for_public_lr0
> +
> +# Create multiple gateway ports.  chassisresident port should not be
> +# created for 'public-lr0' even if there is no localnet port on 'public'
> +# logical switch.
> +check ovn-nbctl --wait=sb lrp-set-gateway-chassis lr0-sw0 gw1
> +# check that there is no port binding cr-public-lr0
> +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +ovn-sbctl dump-flows lr0 > lr0flows
> +ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +AT_CLEANUP
> +])
> diff --git a/tests/ovn.at b/tests/ovn.at
> index 582ec56485..c468673f0c 100644
> --- a/tests/ovn.at
> +++ b/tests/ovn.at
> @@ -21227,10 +21227,10 @@ ovn-nbctl lsp-add sw0 rp-sw0 -- set Logical_Switch_Port rp-sw0 \
>      type=router options:router-port=sw0 \
>      -- lsp-set-addresses rp-sw0 router
>  
> -ovn-nbctl lrp-add lr0 sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
> -    -- lrp-set-gateway-chassis sw1 hv2
> +ovn-nbctl lrp-add lr0 lr0-sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
> +    -- lrp-set-gateway-chassis lr0-sw1 hv2
>  ovn-nbctl lsp-add sw1 rp-sw1 -- set Logical_Switch_Port rp-sw1 \
> -    type=router options:router-port=sw1 \
> +    type=router options:router-port=lr0-sw1 \
>      -- lsp-set-addresses rp-sw1 router
>  
>  ovn-nbctl lsp-add sw0 sw0-p0 \
> @@ -21242,6 +21242,8 @@ ovn-nbctl lsp-add sw0 sw0-p1 \
>  ovn-nbctl lsp-add sw1 sw1-p0 \
>      -- lsp-set-addresses sw1-p0 unknown
>  
> +check ovn-nbctl lsp-add sw1 ln-sw1 -- lsp-set-type ln-sw1 localnet
> +
>  ovn-nbctl lr-nat-add lr0 snat 172.16.1.1 192.168.1.0/24
>  ovn-nbctl lr-nat-add lr0 snat 2002::1 2001::/64
>  
> -- 
> 2.45.2
> 
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Han Zhou Aug. 6, 2024, 6:49 a.m. UTC | #3 
On Mon, Jul 29, 2024 at 7:39 PM <numans@ovn.org> wrote:
>
> From: Numan Siddique <numans@ovn.org>
>
> Consider a deployment with the below logical resources:
>
> 1. A bridged logical switch 'public' with a port - P1 and a localnet
>    port ln-public.
> 2. A logical router 'R'
> 3. Logical switch 'public' connected to R via logical switch/router port
>    peers (public-R and R-public).
> 4. R-public is distributed gateway port with its network as 172.16.0.0/24
> 5. NATs (dnat_and_snat) configured in 'R'.
> 6. And a few overlay logical switches S1, S2 to R.
>
> Any traffic from logical port - P1 of public logical switch destined to
> S1 or S2's logical ports goes out of the source chassis
> (where P1 resides) via the localnet port and reaches the gateway chassis
> which handles the routing.
>
> There are couple of traffic flow scenarios which doesn't work if the
> logical switch 'public' doesn't have a localnet port.
>
> 1. Traffic from port - P1 destined to logical switches S1 or S2 gets
>    dropped in the source chassis.  The packet enters the router R's
>    pipeline, but it gets dropped in the 'lr_in_admission' stage since
>    the logical flow to allow traffic destined to the distributed gateway
>    port MAC is installed only on the gateway chassis.
>
> 2. NAT doesn't work as expected.
>
> In order to suppose this use case (of a logical switch not having a
> localnet port, but has a distributed gateway port and NATs), this patch
> supports the option 'centralize_routing', which can be configured on
> the distributed gateway port (R-public in the example above).
> If this option is set, then routing is centralized on the gateway
> chassis for the traffic destined to the R-public's networks
> (172.16.0.0/24 for the above example).  Traffic from P1 will be
> tunnelled to the gateway chassis.
>
> ovn-northd creates a chassisresident port (cr-public-R) for the
> logical switch port - public-R, along with cr-R-public inorder to
> centralize the traffic.
>
Hi Numan, I have two questions here:

> This feature gets enabled for the distributed gateway port R-public if
>   - The above option is set to true in the R-public's options column.

Why do we need an option to specify this behavior? If the LRP is a
distributed gateway port and there is no localnet port on the
connected LS, can't we just do the centralized routing?

>   - The logical switch 'public' doesn't have any localnet ports.
>   - And R-public is the only distributed gateway port of R.

Why do we require that the R-public is the only distributed gateway
port of R? What if S1 and S2 in this example also connect to R with
distributed router ports? Can we make it work the same way? Or is it
just an initial version and to be extended to support any number of
DGPs in the future?

Best regards,
Han

>
> Distributed NAT (i.e if external_mac and router_port is set) is
> not supported and instead the router port mac is used for such traffic
> and centralized on the gateway chassis.
>
> Reported-at: https://issues.redhat.com/browse/FDP-364
> Acked-by: Mark Michelson <mmichels@redhat.com>
> Signed-off-by: Numan Siddique <numans@ovn.org>
> ---
>
> v1 -> v2
> -------
>    * Corrected the NEWS item entry for the new option.
>    * Rebased and resolved conflicts.
>
> Note: This patch is the continuation from this series - https://patchwork.ozlabs.org/project/ovn/patch/20240606214432.168750-1-numans@ovn.org/
>       Resetted the version number since the new option changed from LSP
>       to LRP.
>
>
>  NEWS                      |   3 +
>  controller/physical.c     |   4 +
>  northd/northd.c           | 257 +++++++++++++++----
>  northd/northd.h           |   1 +
>  ovn-nb.xml                |  34 +++
>  tests/multinode-macros.at |   2 +-
>  tests/multinode.at        | 177 +++++++++++++
>  tests/ovn-northd.at       | 516 +++++++++++++++++++++++++++++++++++++-
>  tests/ovn.at              |   8 +-
>  9 files changed, 952 insertions(+), 50 deletions(-)
>
> diff --git a/NEWS b/NEWS
> index 87e326f21e..8440a74677 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -47,6 +47,9 @@ Post v24.03.0
>    - Add support for CT zone limit that can be specified per LR
>      (options:ct-zone-limit), LS (other_config:ct-zone-limit) or LSP
>      (options:ct-zone-limit).
> +  - A new LRP option 'centralize_routing' has been added to a
> +    distributed gateway port to centralize routing if the logical
> +    switch of its peer doesn't have a localnet port.
>
>  OVN v24.03.0 - 01 Mar 2024
>  --------------------------
> diff --git a/controller/physical.c b/controller/physical.c
> index 3c0200c383..9e04ad5f22 100644
> --- a/controller/physical.c
> +++ b/controller/physical.c
> @@ -1610,6 +1610,10 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name,
>                                                      ct_zones);
>              put_zones_ofpacts(&zone_ids, ofpacts_p);
>
> +            /* Clear the MFF_INPORT.  Its possible that the same packet may
> +             * go out from the same tunnel inport. */
> +            put_load(ofp_to_u16(OFPP_NONE), MFF_IN_PORT, 0, 16, ofpacts_p);
> +
>              /* Resubmit to table 41. */
>              put_resubmit(OFTABLE_CHECK_LOOPBACK, ofpacts_p);
>          }
> diff --git a/northd/northd.c b/northd/northd.c
> index 5c2fd74ff1..4f59d4f1a3 100644
> --- a/northd/northd.c
> +++ b/northd/northd.c
> @@ -2107,6 +2107,55 @@ parse_lsp_addrs(struct ovn_port *op)
>      }
>  }
>
> +static struct ovn_port *
> +create_cr_port(struct ovn_port *op, struct hmap *ports,
> +               struct ovs_list *both_dbs, struct ovs_list *nb_only)
> +{
> +    char *redirect_name = ovn_chassis_redirect_name(
> +        op->nbsp ? op->nbsp->name : op->nbrp->name);
> +
> +    struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> +    if (crp && crp->sb && crp->sb->datapath == op->od->sb) {
> +        ovn_port_set_nb(crp, NULL, op->nbrp);
> +        ovs_list_remove(&crp->list);
> +        ovs_list_push_back(both_dbs, &crp->list);
> +    } else {
> +        crp = ovn_port_create(ports, redirect_name,
> +                              op->nbsp, op->nbrp, NULL);
> +        ovs_list_push_back(nb_only, &crp->list);
> +    }
> +
> +    crp->primary_port = op;
> +    op->cr_port = crp;
> +    crp->od = op->od;
> +    free(redirect_name);
> +
> +    return crp;
> +}
> +
> +/* Returns true if chassis resident port needs to be created for
> + * op's peer logical switch.  False otherwise.
> + *
> + * Chassis resident port needs to be created if the following
> + * conditionsd are met:
> + *   - op is a distributed gateway port
> + *   - op has the option 'centralize_routing' set to true
> + *   - op is the only distributed gateway port attached to its
> + *     router
> + *   - op's peer logical switch has no localnet ports.
> + */
> +static bool
> +peer_needs_cr_port_creation(struct ovn_port *op)
> +{
> +    if ((op->nbrp->n_gateway_chassis || op->nbrp->ha_chassis_group)
> +        && op->od->n_l3dgw_ports == 1 && op->peer && op->peer->nbsp
> +        && !op->peer->od->n_localnet_ports) {
> +        return smap_get_bool(&op->nbrp->options, "centralize_routing", false);
> +    }
> +
> +    return false;
> +}
> +
>  static void
>  join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                     struct hmap *ls_datapaths, struct hmap *lr_datapaths,
> @@ -2214,9 +2263,10 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>              tag_alloc_add_existing_tags(tag_alloc_table, nbsp);
>          }
>      }
> +
> +    struct hmapx dgps = HMAPX_INITIALIZER(&dgps);
>      HMAP_FOR_EACH (od, key_node, lr_datapaths) {
>          ovs_assert(od->nbr);
> -        size_t n_allocated_l3dgw_ports = 0;
>          for (size_t i = 0; i < od->nbr->n_ports; i++) {
>              const struct nbrec_logical_router_port *nbrp
>                  = od->nbr->ports[i];
> @@ -2280,10 +2330,7 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                      redirect_type && !strcasecmp(redirect_type, "bridged");
>              }
>
> -            if (op->nbrp->ha_chassis_group ||
> -                op->nbrp->n_gateway_chassis) {
> -                /* Additional "derived" ovn_port crp represents the
> -                 * instance of op on the gateway chassis. */
> +            if (op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis) {
>                  const char *gw_chassis = smap_get(&op->od->nbr->options,
>                                                 "chassis");
>                  if (gw_chassis) {
> @@ -2292,34 +2339,9 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                      VLOG_WARN_RL(&rl, "Bad configuration: distributed "
>                                   "gateway port configured on port %s "
>                                   "on L3 gateway router", nbrp->name);
> -                    continue;
> -                }
> -
> -                char *redirect_name =
> -                    ovn_chassis_redirect_name(nbrp->name);
> -                struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> -                if (crp && crp->sb && crp->sb->datapath == od->sb) {
> -                    ovn_port_set_nb(crp, NULL, nbrp);
> -                    ovs_list_remove(&crp->list);
> -                    ovs_list_push_back(both, &crp->list);
>                  } else {
> -                    crp = ovn_port_create(ports, redirect_name,
> -                                          NULL, nbrp, NULL);
> -                    ovs_list_push_back(nb_only, &crp->list);
> -                }
> -                crp->primary_port = op;
> -                op->cr_port = crp;
> -                crp->od = od;
> -                free(redirect_name);
> -
> -                /* Add to l3dgw_ports in od, for later use during flow
> -                 * creation. */
> -                if (od->n_l3dgw_ports == n_allocated_l3dgw_ports) {
> -                    od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> -                                                 &n_allocated_l3dgw_ports,
> -                                                 sizeof *od->l3dgw_ports);
> +                    hmapx_add(&dgps, op);
>                  }
> -                od->l3dgw_ports[od->n_l3dgw_ports++] = op;
>             }
>          }
>      }
> @@ -2376,12 +2398,6 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>                          arp_proxy, op->nbsp->name);
>                  }
>              }
> -
> -            /* Only used for the router type LSP whose peer is l3dgw_port */
> -            if (op->peer && is_l3dgw_port(op->peer)) {
> -                op->enable_router_port_acl = smap_get_bool(
> -                    &op->nbsp->options, "enable_router_port_acl", false);
> -            }
>          } else if (op->nbrp && op->nbrp->peer && !is_cr_port(op)) {
>              struct ovn_port *peer = ovn_port_find(ports, op->nbrp->peer);
>              if (peer) {
> @@ -2402,6 +2418,57 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
>          }
>      }
>
> +    struct hmapx_node *hmapx_node;
> +    HMAPX_FOR_EACH (hmapx_node, &dgps) {
> +        op = hmapx_node->data;
> +        od = op->od;
> +        ovs_assert(op->nbrp);
> +        ovs_assert(op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis);
> +
> +        /* Additional "derived" ovn_port crp represents the instance of op on
> +         * the gateway chassis. */
> +        struct ovn_port *crp = create_cr_port(op, ports, both, nb_only);
> +        ovs_assert(crp);
> +
> +        /* Add to l3dgw_ports in od, for later use during flow creation. */
> +        if (od->n_l3dgw_ports == od->n_allocated_l3dgw_ports) {
> +            od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> +                                        &od->n_allocated_l3dgw_ports,
> +                                        sizeof *od->l3dgw_ports);
> +        }
> +        od->l3dgw_ports[od->n_l3dgw_ports++] = op;
> +
> +        if (op->peer && op->peer->nbsp) {
> +            /* Only used for the router type LSP whose peer is l3dgw_port */
> +            op->peer->enable_router_port_acl = smap_get_bool(
> +                    &op->peer->nbsp->options, "enable_router_port_acl", false);
> +        }
> +    }
> +
> +
> +    /* Create chassisresident port for the distributed gateway port's (DGP)
> +     * peer if
> +     *  - DGP's router has only one DGP and
> +     *  - Its peer is a logical switch port and
> +     *  - It's peer's logical switch has no localnet ports and
> +     *  - option 'centralize_routing' is set to true for the DGP.
> +     *
> +     * This is required to support
> +     *   - NAT via geneve (for the overlay provider networks) and
> +     *   - to centralize routing on the gateway chassis for the traffic
> +     *     destined to the DGP's networks.
> +     *
> +     * Future enhancement: Support 'centralizerouting' for all the DGP's
> +     * of a logical router.
> +     * */
> +    HMAPX_FOR_EACH (hmapx_node, &dgps) {
> +        op = hmapx_node->data;
> +        if (peer_needs_cr_port_creation(op)) {
> +            create_cr_port(op->peer, ports, both, nb_only);
> +        }
> +    }
> +    hmapx_destroy(&dgps);
> +
>      /* Wait until all ports have been connected to add to IPAM since
>       * it relies on proper peers to be set
>       */
> @@ -3184,16 +3251,28 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn,
>               * type "l3gateway". */
>              if (chassis) {
>                  sbrec_port_binding_set_type(op->sb, "l3gateway");
> +            } else if (is_cr_port(op)) {
> +                sbrec_port_binding_set_type(op->sb, "chassisredirect");
> +                ovs_assert(op->primary_port->peer);
> +                ovs_assert(op->primary_port->peer->cr_port);
> +                ovs_assert(op->primary_port->peer->cr_port->sb);
> +                sbrec_port_binding_set_ha_chassis_group(
> +                    op->sb,
> +                    op->primary_port->peer->cr_port->sb->ha_chassis_group);
> +
>              } else {
>                  sbrec_port_binding_set_type(op->sb, "patch");
>              }
>
>              const char *router_port = smap_get(&op->nbsp->options,
>                                                 "router-port");
> -            if (router_port || chassis) {
> +            if (router_port || chassis || is_cr_port(op)) {
>                  struct smap new;
>                  smap_init(&new);
> -                if (router_port) {
> +
> +                if (is_cr_port(op)) {
> +                    smap_add(&new, "distributed-port", op->nbsp->name);
> +                } else if (router_port) {
>                      smap_add(&new, "peer", router_port);
>                  }
>                  if (chassis) {
> @@ -8155,9 +8234,27 @@ build_lswitch_rport_arp_req_flow(
>      struct lflow_ref *lflow_ref)
>  {
>      struct ds match   = DS_EMPTY_INITIALIZER;
> +    struct ds m       = DS_EMPTY_INITIALIZER;
>      struct ds actions = DS_EMPTY_INITIALIZER;
>
> -    arp_nd_ns_match(ips, addr_family, &match);
> +    arp_nd_ns_match(ips, addr_family, &m);
> +    ds_clone(&match, &m);
> +
> +    bool has_cr_port = patch_op->cr_port;
> +
> +    /* If the patch_op has a chassis resident port, it means
> +     *    - its peer is a distributed gateway port (DGP) and
> +     *    - routing is centralized for the DGP's networks on
> +     *      the configured gateway chassis.
> +     *
> +     * If that's the case, make sure that the packets destined to
> +     * the DGP's MAC are sent to the chassis where the DGP resides.
> +     * */
> +
> +    if (has_cr_port) {
> +        ds_put_format(&match, " && is_chassis_resident(%s)",
> +                      patch_op->cr_port->json_key);
> +    }
>
>      /* Send a the packet to the router pipeline.  If the switch has non-router
>       * ports then flood it there as well.
> @@ -8179,6 +8276,31 @@ build_lswitch_rport_arp_req_flow(
>                                  lflow_ref);
>      }
>
> +    if (has_cr_port) {
> +        ds_clear(&match);
> +        ds_put_format(&match, "%s && !is_chassis_resident(%s)", ds_cstr(&m),
> +                      patch_op->cr_port->json_key);
> +        ds_clear(&actions);
> +        if (od->n_router_ports != od->nbs->n_ports) {
> +            ds_put_format(&actions, "clone {outport = %s; output; }; "
> +                                    "outport = \""MC_FLOOD_L2"\"; output;",
> +                          patch_op->cr_port->json_key);
> +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> +                                    priority, ds_cstr(&match),
> +                                    ds_cstr(&actions), stage_hint,
> +                                    lflow_ref);
> +        } else {
> +            ds_put_format(&actions, "outport = %s; output;",
> +                          patch_op->cr_port->json_key);
> +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> +                                    priority, ds_cstr(&match),
> +                                    ds_cstr(&actions),
> +                                    stage_hint,
> +                                    lflow_ref);
> +        }
> +    }
> +
> +    ds_destroy(&m);
>      ds_destroy(&match);
>      ds_destroy(&actions);
>  }
> @@ -9548,7 +9670,11 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
>                                  struct ds *actions, struct ds *match)
>  {
>      ovs_assert(op->nbsp);
> -    if (lsp_is_external(op->nbsp)) {
> +
> +    /* Note: A switch port can also have a chassis resident derived port.
> +     * Check if 'op' is a chassis resident dervied port. If so, skip
> +     * adding unicast lookup flows for this port. */
> +    if (lsp_is_external(op->nbsp) || is_cr_port(op)) {
>          return;
>      }
>
> @@ -9566,8 +9692,6 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
>                             "outport = \""MC_UNKNOWN "\"; output;"
>                           : "outport = %s; output;")
>                           : debug_drop_action();
> -    ds_clear(actions);
> -    ds_put_format(actions, action, op->json_key);
>
>      if (lsp_is_router(op->nbsp) && op->peer && op->peer->nbrp) {
>          /* For ports connected to logical routers add flows to bypass the
> @@ -9614,14 +9738,43 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
>              if (add_chassis_resident_check) {
>                  ds_put_format(match, " && is_chassis_resident(%s)", json_key);
>              }
> +        } else if (op->cr_port) {
> +            /* If the op has a chassis resident port, it means
> +             *   - its peer is a distributed gateway port (DGP) and
> +             *   - routing is centralized for the DGP's networks on
> +             *     the configured gateway chassis.
> +             *
> +             * If that's the case, make sure that the packets destined to
> +             * the DGP's MAC are sent to the chassis where the DGP resides.
> +             * */
> +            ds_clear(actions);
> +            ds_put_format(actions, action, op->cr_port->json_key);
> +
> +            struct ds m = DS_EMPTY_INITIALIZER;
> +            ds_put_format(&m, "eth.dst == %s && !is_chassis_resident(%s)",
> +                          op->peer->lrp_networks.ea_s,
> +                          op->cr_port->json_key);
> +
> +            ovn_lflow_add_with_hint(lflows, op->od,
> +                                    S_SWITCH_IN_L2_LKUP, 50,
> +                                    ds_cstr(&m), ds_cstr(actions),
> +                                    &op->nbsp->header_,
> +                                    op->lflow_ref);
> +            ds_destroy(&m);
> +            ds_put_format(match, " && is_chassis_resident(%s)",
> +                          op->cr_port->json_key);
>          }
>
> +        ds_clear(actions);
> +        ds_put_format(actions, action, op->json_key);
>          ovn_lflow_add_with_hint(lflows, op->od,
>                                  S_SWITCH_IN_L2_LKUP, 50,
>                                  ds_cstr(match), ds_cstr(actions),
>                                  &op->nbsp->header_,
>                                  op->lflow_ref);
>      } else {
> +        ds_clear(actions);
> +        ds_put_format(actions, action, op->json_key);
>          for (size_t i = 0; i < op->n_lsp_addrs; i++) {
>              ds_clear(match);
>              ds_put_format(match, "eth.dst == %s", op->lsp_addrs[i].ea_s);
> @@ -11725,6 +11878,14 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op,
>          return;
>      }
>
> +    if (op->peer && op->peer->cr_port) {
> +        /* We don't add the below flows if the router port's peer has
> +         * a chassisresident port.  That's because routing is centralized on
> +         * the gateway chassis for the router port networks/subnets.
> +         */
> +        return;
> +    }
> +
>      /* Mac address to use when replying to ARP/NS. */
>      const char *mac_s = REG_INPORT_ETH_ADDR;
>      struct eth_addr mac;
> @@ -15109,6 +15270,16 @@ lrouter_check_nat_entry(const struct ovn_datapath *od,
>      /* For distributed router NAT, determine whether this NAT rule
>       * satisfies the conditions for distributed NAT processing. */
>      *distributed = false;
> +
> +    /* NAT cannnot be distributed if the DGP's peer
> +     * has a chassisresident port (as the routing is centralized
> +     * on the gateway chassis for the DGP's networks/subnets.)
> +     */
> +    struct ovn_port *l3dgw_port = *nat_l3dgw_port;
> +    if (l3dgw_port && l3dgw_port->peer && l3dgw_port->peer->cr_port) {
> +        return 0;
> +    }
> +
>      if (od->n_l3dgw_ports && !strcmp(nat->type, "dnat_and_snat") &&
>          nat->logical_port && nat->external_mac) {
>          if (eth_addr_from_string(nat->external_mac, mac)) {
> diff --git a/northd/northd.h b/northd/northd.h
> index d4a8d75abc..d7c9655916 100644
> --- a/northd/northd.h
> +++ b/northd/northd.h
> @@ -325,6 +325,7 @@ struct ovn_datapath {
>       * will be NULL. */
>      struct ovn_port **l3dgw_ports;
>      size_t n_l3dgw_ports;
> +    size_t n_allocated_l3dgw_ports;
>
>      /* router datapath has a logical port with redirect-type set to bridged. */
>      bool redirect_bridged;
> diff --git a/ovn-nb.xml b/ovn-nb.xml
> index a4362a4ef1..217d1cd1fe 100644
> --- a/ovn-nb.xml
> +++ b/ovn-nb.xml
> @@ -3499,6 +3499,40 @@ or
>            <ref column="options" key="gateway_mtu"/> option.
>          </p>
>        </column>
> +
> +      <column name="options" key="centralize_routing"
> +              type='{"type": "boolean"}'>
> +        <p>
> +          This option is applicable only if the router port is a
> +          distributed gateway port i.e if the <ref table="Logical_Router_Port"
> +          column="ha_chassis_group"/> column or
> +          <ref table="Logical_Router_Port" column="gateway_chassis"/>
> +          is set.
> +        </p>
> +
> +        <p>
> +          If set to <code>true</code>, routing for the router port's
> +          networks (set in the column <ref table="Logical_Router_Port"
> +          column="networks"/>) is centralized on the gateway chassis
> +          which claims this distributed gateway port.
> +        </p>
> +
> +        <p>
> +          Additionally for this option to take effect, below conditions
> +          must be met:
> +        </p>
> +
> +        <ul>
> +          <li>
> +            The Logical router has only one distributed gateway port.
> +          </li>
> +
> +          <li>
> +            The router port's peer logical switch has no localnet ports.
> +          </li>
> +
> +        </ul>
> +      </column>
>      </group>
>
>      <group title="Attachment">
> diff --git a/tests/multinode-macros.at b/tests/multinode-macros.at
> index 786e564860..757917626c 100644
> --- a/tests/multinode-macros.at
> +++ b/tests/multinode-macros.at
> @@ -92,7 +92,7 @@ m_count_rows() {
>  m_check_row_count() {
>      local db=$(parse_db $1) table=$(parse_table $1); shift
>      local count=$1; shift
> -    local found=$(m_count_rows $c $db:$table "$@")
> +    local found=$(m_count_rows $db:$table "$@")
>      echo
>      echo "Checking for $count rows in $db $table${1+ with $*}... found $found"
>      if test "$count" != "$found"; then
> diff --git a/tests/multinode.at b/tests/multinode.at
> index a7231130ac..a725414165 100644
> --- a/tests/multinode.at
> +++ b/tests/multinode.at
> @@ -1033,6 +1033,183 @@ M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [sh -c 'dd bs=512 count=2 if=/dev/uran
>  done
>  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ip route get 10.0.0.1 dev sw0p1 | grep -q 'mtu 942'])
>
> +# Reset back to geneve tunnels
> +for c in ovn-chassis-1 ovn-chassis-2 ovn-gw-1
> +do
> +    m_as $c ovs-vsctl set open . external-ids:ovn-encap-type=geneve
> +done
> +
> +AT_CLEANUP
> +
> +AT_SETUP([ovn multinode NAT on a provider network with no localnet ports])
> +
> +# Check that ovn-fake-multinode setup is up and running
> +check_fake_multinode_setup
> +
> +# Delete the multinode NB and OVS resources before starting the test.
> +cleanup_multinode_resources
> +
> +check multinode_nbctl ls-add sw0
> +check multinode_nbctl lsp-add sw0 sw0-port1
> +check multinode_nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03 10.0.0.3 1000::3"
> +check multinode_nbctl lsp-add sw0 sw0-port2
> +check multinode_nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:04 10.0.0.4 1000::4"
> +
> +m_as ovn-chassis-1 /data/create_fake_vm.sh sw0-port1 sw0p1 50:54:00:00:00:03 10.0.0.3 24 10.0.0.1 1000::3/64 1000::a
> +m_as ovn-chassis-2 /data/create_fake_vm.sh sw0-port2 sw0p2 50:54:00:00:00:04 10.0.0.4 24 10.0.0.1 1000::4/64 1000::a
> +
> +m_wait_for_ports_up
> +
> +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.4 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +# Create the second logical switch with one port
> +check multinode_nbctl ls-add sw1
> +check multinode_nbctl lsp-add sw1 sw1-port1
> +check multinode_nbctl lsp-set-addresses sw1-port1 "40:54:00:00:00:03 20.0.0.3 2000::3"
> +
> +# Create a logical router and attach both logical switches
> +check multinode_nbctl lr-add lr0
> +check multinode_nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 1000::a/64
> +check multinode_nbctl lsp-add sw0 sw0-lr0
> +check multinode_nbctl lsp-set-type sw0-lr0 router
> +check multinode_nbctl lsp-set-addresses sw0-lr0 router
> +check multinode_nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> +
> +check multinode_nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:02 20.0.0.1/24 2000::a/64
> +check multinode_nbctl lsp-add sw1 sw1-lr0
> +check multinode_nbctl lsp-set-type sw1-lr0 router
> +check multinode_nbctl lsp-set-addresses sw1-lr0 router
> +check multinode_nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> +
> +m_as ovn-chassis-2 /data/create_fake_vm.sh sw1-port1 sw1p1 40:54:00:00:00:03 20.0.0.3 24 20.0.0.1 2000::3/64 2000::a
> +
> +# create exteranl connection for N/S traffic
> +check multinode_nbctl ls-add public
> +check multinode_nbctl lsp-add public ln-public
> +check multinode_nbctl lsp-set-type ln-public localnet
> +check multinode_nbctl lsp-set-addresses ln-public unknown
> +check multinode_nbctl lsp-set-options ln-public network_name=public
> +
> +check multinode_nbctl lrp-add lr0 lr0-public 00:11:22:00:ff:01 172.20.0.100/24
> +check multinode_nbctl lsp-add public public-lr0
> +check multinode_nbctl lsp-set-type public-lr0 router
> +check multinode_nbctl lsp-set-addresses public-lr0 router
> +check multinode_nbctl lsp-set-options public-lr0 router-port=lr0-public
> +check multinode_nbctl lrp-set-gateway-chassis lr0-public ovn-gw-1 10
> +
> +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
> +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.120 20.0.0.3
> +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 10.0.0.0/24
> +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 20.0.0.0/24
> +
> +# Create a logical port pub-p1 and bind it in ovn-chassis-1
> +check multinode_nbctl lsp-add public public-port1
> +check multinode_nbctl lsp-set-addresses public-port1 "60:54:00:00:00:03 172.168.0.50"
> +
> +m_as ovn-chassis-1 /data/create_fake_vm.sh public-port1 pubp1 60:54:00:00:00:03 172.20.0.50 24 172.20.0.100
> +
> +check multinode_nbctl --wait=hv sync
> +
> +# First do basic ping tests before deleting the localnet port - ln-public.
> +# Once the localnet port is deleted from public ls, routing for 172.20.0.0/24
> +# is centralized on ovn-gw-1.
> +
> +# This function checks the North-South traffic.
> +run_ns_traffic() {
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.110], [ignore], [ignore])
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.120], [ignore], [ignore])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-2], [sw1p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  # Now ping from pubp1 to 172.20.0.100, 172.20.0.110, 172.20.0.120, 10.0.0.3 and 20.0.0.3
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.3 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 20.0.0.3 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +}
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
> +# Delete the localnet port by changing the type of ln-public to VIF port.
> +check multinode_nbctl --wait=hv lsp-set-type ln-public ""
> +
> +# cr-port should not be created for public-lr0 since the option
> +# centralize_routing=true is not yet set for lr0-public.
> +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +# Set the option - centralize_routing now.
> +check multinode_nbctl --wait=hv set logical_router_port lr0-public options:centralize_routing=true
> +
> +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> +m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
> +# Re-add the localnet port
> +check multinode_nbctl --wait=hv lsp-set-type ln-public localnet
> +
> +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
> +# Delete the ln-public port this time.
> +check multinode_nbctl --wait=hv lsp-del ln-public
> +
> +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> +m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> +
> +# Test out the N-S traffic.
> +run_ns_traffic
> +
>  AT_CLEANUP
>
>  AT_SETUP([ovn provider network - always_tunnel])
> diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
> index 57f89a7746..649c20f285 100644
> --- a/tests/ovn-northd.at
> +++ b/tests/ovn-northd.at
> @@ -2187,7 +2187,7 @@ match=(inport == "lrp-public" && arp.op == 1 && arp.tpa == 43.43.43.4 && is_chas
>  action=(eth.dst = eth.src; eth.src = 00:00:00:00:00:02; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 00:00:00:00:00:02; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
>  ])
>
> -# xreg0[0..47] isn't used anywhere else.
> +# xreg0[[0..47]] isn't used anywhere else.
>  AT_CHECK([ovn-sbctl lflow-list | grep "xreg0\[[0..47\]]" | grep -vE 'lr_in_admission|lr_in_ip_input'], [1], [])
>
>  AT_CLEANUP
> @@ -5503,13 +5503,14 @@ AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | grep "192.168.4.100" | grep "_MC_flo
>
>  AS_BOX([Configuring ro1-ls1 router port as a gateway router port])
>
> -ovn-nbctl --wait=sb lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> +check ovn-nbctl  lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> +check ovn-nbctl --wait=sb lsp-add ls1 ln-ls1 -- lsp-set-type ln-ls1 localnet
>
>  ovn-sbctl lflow-list ls1 > ls1_lflows
>  AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | ovn_strip_lflows], [0], [dnl
>    table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
>    table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> -  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01), action=(outport = "ls1-ro1"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01 && is_chassis_resident("cr-ro1-ls1")), action=(outport = "ls1-ro1"; output;)
>    table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:02), action=(outport = "vm1"; output;)
>    table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
>    table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:01:01} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> @@ -12475,3 +12476,512 @@ AT_CHECK([ovn-sbctl dump-flows lr | grep lr_in_dnat | ovn_strip_lflows], [0], [d
>
>  AT_CLEANUP
>  ])
> +
> +OVN_FOR_EACH_NORTHD_NO_HV([
> +AT_SETUP([NAT on a provider network with no localnet ports])
> +AT_KEYWORDS([NAT])
> +ovn_start
> +
> +check ovn-nbctl -- ls-add sw0 -- ls-add sw1
> +check ovn-nbctl lsp-add sw0 sw0-port1
> +check ovn-nbctl lr-add lr0
> +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
> +check ovn-nbctl lsp-add sw0 sw0-lr0
> +check ovn-nbctl lsp-set-type sw0-lr0 router
> +check ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01
> +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> +
> +check ovn-nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:03 20.0.0.1/24
> +check ovn-nbctl lsp-add sw1 sw1-lr0
> +check ovn-nbctl lsp-set-type sw1-lr0 router
> +check ovn-nbctl lsp-set-addresses sw1-lr0 router
> +check ovn-nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> +
> +check ovn-sbctl chassis-add gw1 geneve 127.0.0.1
> +check ovn-nbctl ls-add public
> +check ovn-nbctl lsp-add public pub-p1
> +
> +# localnet port
> +check ovn-nbctl lsp-add public ln-public
> +check ovn-nbctl lsp-set-type ln-public localnet
> +check ovn-nbctl lsp-set-addresses ln-public unknown
> +check ovn-nbctl lsp-set-options ln-public network_name=public
> +
> +check ovn-nbctl lrp-add lr0 lr0-public 00:00:00:00:ff:02 172.168.0.10/24
> +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1
> +
> +check ovn-nbctl lsp-add public public-lr0
> +check ovn-nbctl lsp-set-type public-lr0 router
> +check ovn-nbctl lsp-set-addresses public-lr0 router
> +check ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public
> +
> +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
> +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.120 20.0.0.3
> +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 10.0.0.0/24
> +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 20.0.0.0/24
> +
> +check ovn-nbctl --wait=sb sync
> +
> +check_flows_no_cr_port_for_public_lr0() {
> +  # check that there is no port binding cr-public-lr0
> +  check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +  ovn-sbctl dump-flows lr0 > lr0flows
> +  ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
> +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +])
> +
> +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +])
> +
> +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +])
> +
> +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +])
> +
> +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> +])
> +
> +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +])
> +
> +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +])
> +
> +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-lr0-public")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +}
> +
> +check_flows_cr_port_for_public_lr0() {
> +  # check that there is port binding cr-public-lr0
> +  check_row_count Port_Binding 1 logical_port=cr-public-lr0
> +  check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> +
> +  ovn-sbctl dump-flows lr0 > lr0flows
> +  ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
> +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> +])
> +
> +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
> +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> +])
> +
> +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +])
> +
> +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +])
> +
> +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> +])
> +
> +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
> +])
> +
> +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +])
> +
> +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +])
> +
> +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-public-lr0")), action=(outport = "cr-public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-public-lr0")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +}
> +
> +# Check that the lflows are as expected when public has localnet port.
> +check_flows_no_cr_port_for_public_lr0
> +
> +# Remove the localnet port from public logical switch.
> +check ovn-nbctl --wait=sb lsp-set-type ln-public ""
> +
> +# Check that the lflows are as expected and there is no cr port
> +# created for "public-lr0"  when public has no localnet port
> +# since public doesn't have the option "overlay_provider_network=true"
> +# set.
> +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +ovn-sbctl dump-flows lr0 > lr0flows
> +ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +
> +# Set the option "centralize_routing=true" for lr0-public.
> +check ovn-nbctl --wait=sb set logical_router_port lr0-public options:centralize_routing=true
> +
> +# Check that the lflows are as expected and there is cr port created for public-lr0.
> +check_flows_cr_port_for_public_lr0
> +
> +# Set the type of ln-public back to localnet
> +check ovn-nbctl --wait=sb lsp-set-type ln-public localnet
> +
> +# Check that the lflows are as expected when public has localnet port.
> +check_flows_no_cr_port_for_public_lr0
> +
> +# Delete the localnet port
> +check ovn-nbctl --wait=sb lsp-del ln-public
> +
> +# Check that the lflows are as expected when public has no localnet port.
> +check_flows_cr_port_for_public_lr0
> +
> +# Create multiple gateway ports.  chassisresident port should not be
> +# created for 'public-lr0' even if there is no localnet port on 'public'
> +# logical switch.
> +check ovn-nbctl --wait=sb lrp-set-gateway-chassis lr0-sw0 gw1
> +# check that there is no port binding cr-public-lr0
> +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> +
> +ovn-sbctl dump-flows lr0 > lr0flows
> +ovn-sbctl dump-flows public > publicflows
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> +])
> +
> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> +])
> +
> +AT_CLEANUP
> +])
> diff --git a/tests/ovn.at b/tests/ovn.at
> index 582ec56485..c468673f0c 100644
> --- a/tests/ovn.at
> +++ b/tests/ovn.at
> @@ -21227,10 +21227,10 @@ ovn-nbctl lsp-add sw0 rp-sw0 -- set Logical_Switch_Port rp-sw0 \
>      type=router options:router-port=sw0 \
>      -- lsp-set-addresses rp-sw0 router
>
> -ovn-nbctl lrp-add lr0 sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
> -    -- lrp-set-gateway-chassis sw1 hv2
> +ovn-nbctl lrp-add lr0 lr0-sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
> +    -- lrp-set-gateway-chassis lr0-sw1 hv2
>  ovn-nbctl lsp-add sw1 rp-sw1 -- set Logical_Switch_Port rp-sw1 \
> -    type=router options:router-port=sw1 \
> +    type=router options:router-port=lr0-sw1 \
>      -- lsp-set-addresses rp-sw1 router
>
>  ovn-nbctl lsp-add sw0 sw0-p0 \
> @@ -21242,6 +21242,8 @@ ovn-nbctl lsp-add sw0 sw0-p1 \
>  ovn-nbctl lsp-add sw1 sw1-p0 \
>      -- lsp-set-addresses sw1-p0 unknown
>
> +check ovn-nbctl lsp-add sw1 ln-sw1 -- lsp-set-type ln-sw1 localnet
> +
>  ovn-nbctl lr-nat-add lr0 snat 172.16.1.1 192.168.1.0/24
>  ovn-nbctl lr-nat-add lr0 snat 2002::1 2001::/64
>
> --
> 2.45.2
>
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Numan Siddique Aug. 7, 2024, 12:06 a.m. UTC | #4 
On Mon, Aug 5, 2024 at 8:45 AM Felix Huettner via dev
<ovs-dev@openvswitch.org> wrote:
>
> On Mon, Jul 29, 2024 at 10:38:49PM -0400, numans@ovn.org wrote:
> > From: Numan Siddique <numans@ovn.org>
> >
> > Consider a deployment with the below logical resources:
> >
> > 1. A bridged logical switch 'public' with a port - P1 and a localnet
> >    port ln-public.
> > 2. A logical router 'R'
> > 3. Logical switch 'public' connected to R via logical switch/router port
> >    peers (public-R and R-public).
> > 4. R-public is distributed gateway port with its network as 172.16.0.0/24
> > 5. NATs (dnat_and_snat) configured in 'R'.
> > 6. And a few overlay logical switches S1, S2 to R.
> >
> > Any traffic from logical port - P1 of public logical switch destined to
> > S1 or S2's logical ports goes out of the source chassis
> > (where P1 resides) via the localnet port and reaches the gateway chassis
> > which handles the routing.
> >
> > There are couple of traffic flow scenarios which doesn't work if the
> > logical switch 'public' doesn't have a localnet port.
> >
> > 1. Traffic from port - P1 destined to logical switches S1 or S2 gets
> >    dropped in the source chassis.  The packet enters the router R's
> >    pipeline, but it gets dropped in the 'lr_in_admission' stage since
> >    the logical flow to allow traffic destined to the distributed gateway
> >    port MAC is installed only on the gateway chassis.
> >
> > 2. NAT doesn't work as expected.
> >
> > In order to suppose this use case (of a logical switch not having a
> > localnet port, but has a distributed gateway port and NATs), this patch
> > supports the option 'centralize_routing', which can be configured on
> > the distributed gateway port (R-public in the example above).
> > If this option is set, then routing is centralized on the gateway
> > chassis for the traffic destined to the R-public's networks
> > (172.16.0.0/24 for the above example).  Traffic from P1 will be
> > tunnelled to the gateway chassis.
> >
> > ovn-northd creates a chassisresident port (cr-public-R) for the
> > logical switch port - public-R, along with cr-R-public inorder to
> > centralize the traffic.
> >
> > This feature gets enabled for the distributed gateway port R-public if
> >   - The above option is set to true in the R-public's options column.
> >   - The logical switch 'public' doesn't have any localnet ports.
> >   - And R-public is the only distributed gateway port of R.
> >
> > Distributed NAT (i.e if external_mac and router_port is set) is
> > not supported and instead the router port mac is used for such traffic
> > and centralized on the gateway chassis.
> >
> > Reported-at: https://issues.redhat.com/browse/FDP-364
> > Acked-by: Mark Michelson <mmichels@redhat.com>
> > Signed-off-by: Numan Siddique <numans@ovn.org>
> > ---
> >
> > v1 -> v2
> > -------
> >    * Corrected the NEWS item entry for the new option.
> >    * Rebased and resolved conflicts.
> >
> > Note: This patch is the continuation from this series - https://patchwork.ozlabs.org/project/ovn/patch/20240606214432.168750-1-numans@ovn.org/
> >       Resetted the version number since the new option changed from LSP
> >       to LRP.
> >
> >
> >  NEWS                      |   3 +
> >  controller/physical.c     |   4 +
> >  northd/northd.c           | 257 +++++++++++++++----
> >  northd/northd.h           |   1 +
> >  ovn-nb.xml                |  34 +++
> >  tests/multinode-macros.at |   2 +-
> >  tests/multinode.at        | 177 +++++++++++++
> >  tests/ovn-northd.at       | 516 +++++++++++++++++++++++++++++++++++++-
> >  tests/ovn.at              |   8 +-
> >  9 files changed, 952 insertions(+), 50 deletions(-)
> >
> > diff --git a/NEWS b/NEWS
> > index 87e326f21e..8440a74677 100644
> > --- a/NEWS
> > +++ b/NEWS
> > @@ -47,6 +47,9 @@ Post v24.03.0
> >    - Add support for CT zone limit that can be specified per LR
> >      (options:ct-zone-limit), LS (other_config:ct-zone-limit) or LSP
> >      (options:ct-zone-limit).
> > +  - A new LRP option 'centralize_routing' has been added to a
> > +    distributed gateway port to centralize routing if the logical
> > +    switch of its peer doesn't have a localnet port.
> >
> >  OVN v24.03.0 - 01 Mar 2024
> >  --------------------------
> > diff --git a/controller/physical.c b/controller/physical.c
> > index 3c0200c383..9e04ad5f22 100644
> > --- a/controller/physical.c
> > +++ b/controller/physical.c
> > @@ -1610,6 +1610,10 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> >                                                      ct_zones);
> >              put_zones_ofpacts(&zone_ids, ofpacts_p);
> >
> > +            /* Clear the MFF_INPORT.  Its possible that the same packet may
> > +             * go out from the same tunnel inport. */
> > +            put_load(ofp_to_u16(OFPP_NONE), MFF_IN_PORT, 0, 16, ofpacts_p);
> > +
> >              /* Resubmit to table 41. */
> >              put_resubmit(OFTABLE_CHECK_LOOPBACK, ofpacts_p);
> >          }
> > diff --git a/northd/northd.c b/northd/northd.c
> > index 5c2fd74ff1..4f59d4f1a3 100644
> > --- a/northd/northd.c
> > +++ b/northd/northd.c
> > @@ -2107,6 +2107,55 @@ parse_lsp_addrs(struct ovn_port *op)
> >      }
> >  }
> >
> > +static struct ovn_port *
> > +create_cr_port(struct ovn_port *op, struct hmap *ports,
> > +               struct ovs_list *both_dbs, struct ovs_list *nb_only)
> > +{
> > +    char *redirect_name = ovn_chassis_redirect_name(
> > +        op->nbsp ? op->nbsp->name : op->nbrp->name);
> > +
> > +    struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> > +    if (crp && crp->sb && crp->sb->datapath == op->od->sb) {
> > +        ovn_port_set_nb(crp, NULL, op->nbrp);
>
> Hi Numan,
>
> i think you need to pass op->nbsp instead of NULL here.
>
> Otherwise i observe the following crash:
>
> northd/northd.c:4371:48: runtime error: member access within null pointer of type 'const struct nbrec_logical_router_port'
> SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior northd/northd.c:4371:48
> AddressSanitizer:DEADLYSIGNAL
> =================================================================
> ==84927==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000180 (pc 0x57ab3854f04a bp 0x7ffc54d1ba30 sp 0x7ffc54d1ba20 T0)
> ==84927==The signal is caused by a READ memory access.
> ==84927==Hint: address points to the zero page.
>     #0 0x57ab3854f04a in hmap_first_with_hash /home/ovn/ovs/./include/openvswitch/hmap.h:360:38
>     #1 0x57ab3854fedf in smap_find__ /home/ovn/ovs/lib/smap.c:421:5
>     #2 0x57ab3854f7b8 in smap_get_node /home/ovn/ovs/lib/smap.c:217:12
>     #3 0x57ab3854f74f in smap_get_def /home/ovn/ovs/lib/smap.c:208:30
>     #4 0x57ab3854f726 in smap_get /home/ovn/ovs/lib/smap.c:200:12
>     #5 0x57ab3854f862 in smap_get_int /home/ovn/ovs/lib/smap.c:240:25
>     #6 0x57ab383222eb in ovn_port_assign_requested_tnl_id /home/ovn/northd/northd.c:4372:27
>     #7 0x57ab383072fa in build_ports /home/ovn/northd/northd.c:4454:9
>     #8 0x57ab38301457 in ovnnb_db_run /home/ovn/northd/northd.c:18023:5
>     #9 0x57ab3841d99e in en_northd_run /home/ovn/northd/en-northd.c:137:5
>     #10 0x57ab384599b2 in engine_recompute /home/ovn/lib/inc-proc-eng.c:411:5
>     #11 0x57ab38459d6e in engine_run_node /home/ovn/lib/inc-proc-eng.c:473:9
>     #12 0x57ab38459ec3 in engine_run /home/ovn/lib/inc-proc-eng.c:524:9
>     #13 0x57ab38430c5d in inc_proc_northd_run /home/ovn/northd/inc-proc-northd.c:420:5
>     #14 0x57ab3841bb2f in main /home/ovn/northd/ovn-northd.c:970:32
>     #15 0x7ab516c2a1c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
>     #16 0x7ab516c2a28a in __libc_start_main csu/../csu/libc-start.c:360:3
>     #17 0x57ab3818cda4 in _start (/usr/local/bin/ovn-northd+0x126da4) (BuildId: 9163c1ae41fe5e3ca08fb7f1741bbea50426b335)
>
> I guess this is because nbrp will be null as the port is derived from a lsp and
> therefor we later would have neither nbrp nor nbsp.

Thanks for pointing this out.  I'll fix it in v3.

Numan

>
> In case the port does not yet exist in the ports hmap it is below
> generated correctly with nbrp and nbsp whatever is set.
>
> Thanks
> Felix
>
> > +        ovs_list_remove(&crp->list);
> > +        ovs_list_push_back(both_dbs, &crp->list);
> > +    } else {
> > +        crp = ovn_port_create(ports, redirect_name,
> > +                              op->nbsp, op->nbrp, NULL);
> > +        ovs_list_push_back(nb_only, &crp->list);
> > +    }
> > +
> > +    crp->primary_port = op;
> > +    op->cr_port = crp;
> > +    crp->od = op->od;
> > +    free(redirect_name);
> > +
> > +    return crp;
> > +}
> > +
> > +/* Returns true if chassis resident port needs to be created for
> > + * op's peer logical switch.  False otherwise.
> > + *
> > + * Chassis resident port needs to be created if the following
> > + * conditionsd are met:
> > + *   - op is a distributed gateway port
> > + *   - op has the option 'centralize_routing' set to true
> > + *   - op is the only distributed gateway port attached to its
> > + *     router
> > + *   - op's peer logical switch has no localnet ports.
> > + */
> > +static bool
> > +peer_needs_cr_port_creation(struct ovn_port *op)
> > +{
> > +    if ((op->nbrp->n_gateway_chassis || op->nbrp->ha_chassis_group)
> > +        && op->od->n_l3dgw_ports == 1 && op->peer && op->peer->nbsp
> > +        && !op->peer->od->n_localnet_ports) {
> > +        return smap_get_bool(&op->nbrp->options, "centralize_routing", false);
> > +    }
> > +
> > +    return false;
> > +}
> > +
> >  static void
> >  join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> >                     struct hmap *ls_datapaths, struct hmap *lr_datapaths,
> > @@ -2214,9 +2263,10 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> >              tag_alloc_add_existing_tags(tag_alloc_table, nbsp);
> >          }
> >      }
> > +
> > +    struct hmapx dgps = HMAPX_INITIALIZER(&dgps);
> >      HMAP_FOR_EACH (od, key_node, lr_datapaths) {
> >          ovs_assert(od->nbr);
> > -        size_t n_allocated_l3dgw_ports = 0;
> >          for (size_t i = 0; i < od->nbr->n_ports; i++) {
> >              const struct nbrec_logical_router_port *nbrp
> >                  = od->nbr->ports[i];
> > @@ -2280,10 +2330,7 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> >                      redirect_type && !strcasecmp(redirect_type, "bridged");
> >              }
> >
> > -            if (op->nbrp->ha_chassis_group ||
> > -                op->nbrp->n_gateway_chassis) {
> > -                /* Additional "derived" ovn_port crp represents the
> > -                 * instance of op on the gateway chassis. */
> > +            if (op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis) {
> >                  const char *gw_chassis = smap_get(&op->od->nbr->options,
> >                                                 "chassis");
> >                  if (gw_chassis) {
> > @@ -2292,34 +2339,9 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> >                      VLOG_WARN_RL(&rl, "Bad configuration: distributed "
> >                                   "gateway port configured on port %s "
> >                                   "on L3 gateway router", nbrp->name);
> > -                    continue;
> > -                }
> > -
> > -                char *redirect_name =
> > -                    ovn_chassis_redirect_name(nbrp->name);
> > -                struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> > -                if (crp && crp->sb && crp->sb->datapath == od->sb) {
> > -                    ovn_port_set_nb(crp, NULL, nbrp);
> > -                    ovs_list_remove(&crp->list);
> > -                    ovs_list_push_back(both, &crp->list);
> >                  } else {
> > -                    crp = ovn_port_create(ports, redirect_name,
> > -                                          NULL, nbrp, NULL);
> > -                    ovs_list_push_back(nb_only, &crp->list);
> > -                }
> > -                crp->primary_port = op;
> > -                op->cr_port = crp;
> > -                crp->od = od;
> > -                free(redirect_name);
> > -
> > -                /* Add to l3dgw_ports in od, for later use during flow
> > -                 * creation. */
> > -                if (od->n_l3dgw_ports == n_allocated_l3dgw_ports) {
> > -                    od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> > -                                                 &n_allocated_l3dgw_ports,
> > -                                                 sizeof *od->l3dgw_ports);
> > +                    hmapx_add(&dgps, op);
> >                  }
> > -                od->l3dgw_ports[od->n_l3dgw_ports++] = op;
> >             }
> >          }
> >      }
> > @@ -2376,12 +2398,6 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> >                          arp_proxy, op->nbsp->name);
> >                  }
> >              }
> > -
> > -            /* Only used for the router type LSP whose peer is l3dgw_port */
> > -            if (op->peer && is_l3dgw_port(op->peer)) {
> > -                op->enable_router_port_acl = smap_get_bool(
> > -                    &op->nbsp->options, "enable_router_port_acl", false);
> > -            }
> >          } else if (op->nbrp && op->nbrp->peer && !is_cr_port(op)) {
> >              struct ovn_port *peer = ovn_port_find(ports, op->nbrp->peer);
> >              if (peer) {
> > @@ -2402,6 +2418,57 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> >          }
> >      }
> >
> > +    struct hmapx_node *hmapx_node;
> > +    HMAPX_FOR_EACH (hmapx_node, &dgps) {
> > +        op = hmapx_node->data;
> > +        od = op->od;
> > +        ovs_assert(op->nbrp);
> > +        ovs_assert(op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis);
> > +
> > +        /* Additional "derived" ovn_port crp represents the instance of op on
> > +         * the gateway chassis. */
> > +        struct ovn_port *crp = create_cr_port(op, ports, both, nb_only);
> > +        ovs_assert(crp);
> > +
> > +        /* Add to l3dgw_ports in od, for later use during flow creation. */
> > +        if (od->n_l3dgw_ports == od->n_allocated_l3dgw_ports) {
> > +            od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> > +                                        &od->n_allocated_l3dgw_ports,
> > +                                        sizeof *od->l3dgw_ports);
> > +        }
> > +        od->l3dgw_ports[od->n_l3dgw_ports++] = op;
> > +
> > +        if (op->peer && op->peer->nbsp) {
> > +            /* Only used for the router type LSP whose peer is l3dgw_port */
> > +            op->peer->enable_router_port_acl = smap_get_bool(
> > +                    &op->peer->nbsp->options, "enable_router_port_acl", false);
> > +        }
> > +    }
> > +
> > +
> > +    /* Create chassisresident port for the distributed gateway port's (DGP)
> > +     * peer if
> > +     *  - DGP's router has only one DGP and
> > +     *  - Its peer is a logical switch port and
> > +     *  - It's peer's logical switch has no localnet ports and
> > +     *  - option 'centralize_routing' is set to true for the DGP.
> > +     *
> > +     * This is required to support
> > +     *   - NAT via geneve (for the overlay provider networks) and
> > +     *   - to centralize routing on the gateway chassis for the traffic
> > +     *     destined to the DGP's networks.
> > +     *
> > +     * Future enhancement: Support 'centralizerouting' for all the DGP's
> > +     * of a logical router.
> > +     * */
> > +    HMAPX_FOR_EACH (hmapx_node, &dgps) {
> > +        op = hmapx_node->data;
> > +        if (peer_needs_cr_port_creation(op)) {
> > +            create_cr_port(op->peer, ports, both, nb_only);
> > +        }
> > +    }
> > +    hmapx_destroy(&dgps);
> > +
> >      /* Wait until all ports have been connected to add to IPAM since
> >       * it relies on proper peers to be set
> >       */
> > @@ -3184,16 +3251,28 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn,
> >               * type "l3gateway". */
> >              if (chassis) {
> >                  sbrec_port_binding_set_type(op->sb, "l3gateway");
> > +            } else if (is_cr_port(op)) {
> > +                sbrec_port_binding_set_type(op->sb, "chassisredirect");
> > +                ovs_assert(op->primary_port->peer);
> > +                ovs_assert(op->primary_port->peer->cr_port);
> > +                ovs_assert(op->primary_port->peer->cr_port->sb);
> > +                sbrec_port_binding_set_ha_chassis_group(
> > +                    op->sb,
> > +                    op->primary_port->peer->cr_port->sb->ha_chassis_group);
> > +
> >              } else {
> >                  sbrec_port_binding_set_type(op->sb, "patch");
> >              }
> >
> >              const char *router_port = smap_get(&op->nbsp->options,
> >                                                 "router-port");
> > -            if (router_port || chassis) {
> > +            if (router_port || chassis || is_cr_port(op)) {
> >                  struct smap new;
> >                  smap_init(&new);
> > -                if (router_port) {
> > +
> > +                if (is_cr_port(op)) {
> > +                    smap_add(&new, "distributed-port", op->nbsp->name);
> > +                } else if (router_port) {
> >                      smap_add(&new, "peer", router_port);
> >                  }
> >                  if (chassis) {
> > @@ -8155,9 +8234,27 @@ build_lswitch_rport_arp_req_flow(
> >      struct lflow_ref *lflow_ref)
> >  {
> >      struct ds match   = DS_EMPTY_INITIALIZER;
> > +    struct ds m       = DS_EMPTY_INITIALIZER;
> >      struct ds actions = DS_EMPTY_INITIALIZER;
> >
> > -    arp_nd_ns_match(ips, addr_family, &match);
> > +    arp_nd_ns_match(ips, addr_family, &m);
> > +    ds_clone(&match, &m);
> > +
> > +    bool has_cr_port = patch_op->cr_port;
> > +
> > +    /* If the patch_op has a chassis resident port, it means
> > +     *    - its peer is a distributed gateway port (DGP) and
> > +     *    - routing is centralized for the DGP's networks on
> > +     *      the configured gateway chassis.
> > +     *
> > +     * If that's the case, make sure that the packets destined to
> > +     * the DGP's MAC are sent to the chassis where the DGP resides.
> > +     * */
> > +
> > +    if (has_cr_port) {
> > +        ds_put_format(&match, " && is_chassis_resident(%s)",
> > +                      patch_op->cr_port->json_key);
> > +    }
> >
> >      /* Send a the packet to the router pipeline.  If the switch has non-router
> >       * ports then flood it there as well.
> > @@ -8179,6 +8276,31 @@ build_lswitch_rport_arp_req_flow(
> >                                  lflow_ref);
> >      }
> >
> > +    if (has_cr_port) {
> > +        ds_clear(&match);
> > +        ds_put_format(&match, "%s && !is_chassis_resident(%s)", ds_cstr(&m),
> > +                      patch_op->cr_port->json_key);
> > +        ds_clear(&actions);
> > +        if (od->n_router_ports != od->nbs->n_ports) {
> > +            ds_put_format(&actions, "clone {outport = %s; output; }; "
> > +                                    "outport = \""MC_FLOOD_L2"\"; output;",
> > +                          patch_op->cr_port->json_key);
> > +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> > +                                    priority, ds_cstr(&match),
> > +                                    ds_cstr(&actions), stage_hint,
> > +                                    lflow_ref);
> > +        } else {
> > +            ds_put_format(&actions, "outport = %s; output;",
> > +                          patch_op->cr_port->json_key);
> > +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> > +                                    priority, ds_cstr(&match),
> > +                                    ds_cstr(&actions),
> > +                                    stage_hint,
> > +                                    lflow_ref);
> > +        }
> > +    }
> > +
> > +    ds_destroy(&m);
> >      ds_destroy(&match);
> >      ds_destroy(&actions);
> >  }
> > @@ -9548,7 +9670,11 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
> >                                  struct ds *actions, struct ds *match)
> >  {
> >      ovs_assert(op->nbsp);
> > -    if (lsp_is_external(op->nbsp)) {
> > +
> > +    /* Note: A switch port can also have a chassis resident derived port.
> > +     * Check if 'op' is a chassis resident dervied port. If so, skip
> > +     * adding unicast lookup flows for this port. */
> > +    if (lsp_is_external(op->nbsp) || is_cr_port(op)) {
> >          return;
> >      }
> >
> > @@ -9566,8 +9692,6 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
> >                             "outport = \""MC_UNKNOWN "\"; output;"
> >                           : "outport = %s; output;")
> >                           : debug_drop_action();
> > -    ds_clear(actions);
> > -    ds_put_format(actions, action, op->json_key);
> >
> >      if (lsp_is_router(op->nbsp) && op->peer && op->peer->nbrp) {
> >          /* For ports connected to logical routers add flows to bypass the
> > @@ -9614,14 +9738,43 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
> >              if (add_chassis_resident_check) {
> >                  ds_put_format(match, " && is_chassis_resident(%s)", json_key);
> >              }
> > +        } else if (op->cr_port) {
> > +            /* If the op has a chassis resident port, it means
> > +             *   - its peer is a distributed gateway port (DGP) and
> > +             *   - routing is centralized for the DGP's networks on
> > +             *     the configured gateway chassis.
> > +             *
> > +             * If that's the case, make sure that the packets destined to
> > +             * the DGP's MAC are sent to the chassis where the DGP resides.
> > +             * */
> > +            ds_clear(actions);
> > +            ds_put_format(actions, action, op->cr_port->json_key);
> > +
> > +            struct ds m = DS_EMPTY_INITIALIZER;
> > +            ds_put_format(&m, "eth.dst == %s && !is_chassis_resident(%s)",
> > +                          op->peer->lrp_networks.ea_s,
> > +                          op->cr_port->json_key);
> > +
> > +            ovn_lflow_add_with_hint(lflows, op->od,
> > +                                    S_SWITCH_IN_L2_LKUP, 50,
> > +                                    ds_cstr(&m), ds_cstr(actions),
> > +                                    &op->nbsp->header_,
> > +                                    op->lflow_ref);
> > +            ds_destroy(&m);
> > +            ds_put_format(match, " && is_chassis_resident(%s)",
> > +                          op->cr_port->json_key);
> >          }
> >
> > +        ds_clear(actions);
> > +        ds_put_format(actions, action, op->json_key);
> >          ovn_lflow_add_with_hint(lflows, op->od,
> >                                  S_SWITCH_IN_L2_LKUP, 50,
> >                                  ds_cstr(match), ds_cstr(actions),
> >                                  &op->nbsp->header_,
> >                                  op->lflow_ref);
> >      } else {
> > +        ds_clear(actions);
> > +        ds_put_format(actions, action, op->json_key);
> >          for (size_t i = 0; i < op->n_lsp_addrs; i++) {
> >              ds_clear(match);
> >              ds_put_format(match, "eth.dst == %s", op->lsp_addrs[i].ea_s);
> > @@ -11725,6 +11878,14 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op,
> >          return;
> >      }
> >
> > +    if (op->peer && op->peer->cr_port) {
> > +        /* We don't add the below flows if the router port's peer has
> > +         * a chassisresident port.  That's because routing is centralized on
> > +         * the gateway chassis for the router port networks/subnets.
> > +         */
> > +        return;
> > +    }
> > +
> >      /* Mac address to use when replying to ARP/NS. */
> >      const char *mac_s = REG_INPORT_ETH_ADDR;
> >      struct eth_addr mac;
> > @@ -15109,6 +15270,16 @@ lrouter_check_nat_entry(const struct ovn_datapath *od,
> >      /* For distributed router NAT, determine whether this NAT rule
> >       * satisfies the conditions for distributed NAT processing. */
> >      *distributed = false;
> > +
> > +    /* NAT cannnot be distributed if the DGP's peer
> > +     * has a chassisresident port (as the routing is centralized
> > +     * on the gateway chassis for the DGP's networks/subnets.)
> > +     */
> > +    struct ovn_port *l3dgw_port = *nat_l3dgw_port;
> > +    if (l3dgw_port && l3dgw_port->peer && l3dgw_port->peer->cr_port) {
> > +        return 0;
> > +    }
> > +
> >      if (od->n_l3dgw_ports && !strcmp(nat->type, "dnat_and_snat") &&
> >          nat->logical_port && nat->external_mac) {
> >          if (eth_addr_from_string(nat->external_mac, mac)) {
> > diff --git a/northd/northd.h b/northd/northd.h
> > index d4a8d75abc..d7c9655916 100644
> > --- a/northd/northd.h
> > +++ b/northd/northd.h
> > @@ -325,6 +325,7 @@ struct ovn_datapath {
> >       * will be NULL. */
> >      struct ovn_port **l3dgw_ports;
> >      size_t n_l3dgw_ports;
> > +    size_t n_allocated_l3dgw_ports;
> >
> >      /* router datapath has a logical port with redirect-type set to bridged. */
> >      bool redirect_bridged;
> > diff --git a/ovn-nb.xml b/ovn-nb.xml
> > index a4362a4ef1..217d1cd1fe 100644
> > --- a/ovn-nb.xml
> > +++ b/ovn-nb.xml
> > @@ -3499,6 +3499,40 @@ or
> >            <ref column="options" key="gateway_mtu"/> option.
> >          </p>
> >        </column>
> > +
> > +      <column name="options" key="centralize_routing"
> > +              type='{"type": "boolean"}'>
> > +        <p>
> > +          This option is applicable only if the router port is a
> > +          distributed gateway port i.e if the <ref table="Logical_Router_Port"
> > +          column="ha_chassis_group"/> column or
> > +          <ref table="Logical_Router_Port" column="gateway_chassis"/>
> > +          is set.
> > +        </p>
> > +
> > +        <p>
> > +          If set to <code>true</code>, routing for the router port's
> > +          networks (set in the column <ref table="Logical_Router_Port"
> > +          column="networks"/>) is centralized on the gateway chassis
> > +          which claims this distributed gateway port.
> > +        </p>
> > +
> > +        <p>
> > +          Additionally for this option to take effect, below conditions
> > +          must be met:
> > +        </p>
> > +
> > +        <ul>
> > +          <li>
> > +            The Logical router has only one distributed gateway port.
> > +          </li>
> > +
> > +          <li>
> > +            The router port's peer logical switch has no localnet ports.
> > +          </li>
> > +
> > +        </ul>
> > +      </column>
> >      </group>
> >
> >      <group title="Attachment">
> > diff --git a/tests/multinode-macros.at b/tests/multinode-macros.at
> > index 786e564860..757917626c 100644
> > --- a/tests/multinode-macros.at
> > +++ b/tests/multinode-macros.at
> > @@ -92,7 +92,7 @@ m_count_rows() {
> >  m_check_row_count() {
> >      local db=$(parse_db $1) table=$(parse_table $1); shift
> >      local count=$1; shift
> > -    local found=$(m_count_rows $c $db:$table "$@")
> > +    local found=$(m_count_rows $db:$table "$@")
> >      echo
> >      echo "Checking for $count rows in $db $table${1+ with $*}... found $found"
> >      if test "$count" != "$found"; then
> > diff --git a/tests/multinode.at b/tests/multinode.at
> > index a7231130ac..a725414165 100644
> > --- a/tests/multinode.at
> > +++ b/tests/multinode.at
> > @@ -1033,6 +1033,183 @@ M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [sh -c 'dd bs=512 count=2 if=/dev/uran
> >  done
> >  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ip route get 10.0.0.1 dev sw0p1 | grep -q 'mtu 942'])
> >
> > +# Reset back to geneve tunnels
> > +for c in ovn-chassis-1 ovn-chassis-2 ovn-gw-1
> > +do
> > +    m_as $c ovs-vsctl set open . external-ids:ovn-encap-type=geneve
> > +done
> > +
> > +AT_CLEANUP
> > +
> > +AT_SETUP([ovn multinode NAT on a provider network with no localnet ports])
> > +
> > +# Check that ovn-fake-multinode setup is up and running
> > +check_fake_multinode_setup
> > +
> > +# Delete the multinode NB and OVS resources before starting the test.
> > +cleanup_multinode_resources
> > +
> > +check multinode_nbctl ls-add sw0
> > +check multinode_nbctl lsp-add sw0 sw0-port1
> > +check multinode_nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03 10.0.0.3 1000::3"
> > +check multinode_nbctl lsp-add sw0 sw0-port2
> > +check multinode_nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:04 10.0.0.4 1000::4"
> > +
> > +m_as ovn-chassis-1 /data/create_fake_vm.sh sw0-port1 sw0p1 50:54:00:00:00:03 10.0.0.3 24 10.0.0.1 1000::3/64 1000::a
> > +m_as ovn-chassis-2 /data/create_fake_vm.sh sw0-port2 sw0p2 50:54:00:00:00:04 10.0.0.4 24 10.0.0.1 1000::4/64 1000::a
> > +
> > +m_wait_for_ports_up
> > +
> > +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.4 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +# Create the second logical switch with one port
> > +check multinode_nbctl ls-add sw1
> > +check multinode_nbctl lsp-add sw1 sw1-port1
> > +check multinode_nbctl lsp-set-addresses sw1-port1 "40:54:00:00:00:03 20.0.0.3 2000::3"
> > +
> > +# Create a logical router and attach both logical switches
> > +check multinode_nbctl lr-add lr0
> > +check multinode_nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 1000::a/64
> > +check multinode_nbctl lsp-add sw0 sw0-lr0
> > +check multinode_nbctl lsp-set-type sw0-lr0 router
> > +check multinode_nbctl lsp-set-addresses sw0-lr0 router
> > +check multinode_nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> > +
> > +check multinode_nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:02 20.0.0.1/24 2000::a/64
> > +check multinode_nbctl lsp-add sw1 sw1-lr0
> > +check multinode_nbctl lsp-set-type sw1-lr0 router
> > +check multinode_nbctl lsp-set-addresses sw1-lr0 router
> > +check multinode_nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> > +
> > +m_as ovn-chassis-2 /data/create_fake_vm.sh sw1-port1 sw1p1 40:54:00:00:00:03 20.0.0.3 24 20.0.0.1 2000::3/64 2000::a
> > +
> > +# create exteranl connection for N/S traffic
> > +check multinode_nbctl ls-add public
> > +check multinode_nbctl lsp-add public ln-public
> > +check multinode_nbctl lsp-set-type ln-public localnet
> > +check multinode_nbctl lsp-set-addresses ln-public unknown
> > +check multinode_nbctl lsp-set-options ln-public network_name=public
> > +
> > +check multinode_nbctl lrp-add lr0 lr0-public 00:11:22:00:ff:01 172.20.0.100/24
> > +check multinode_nbctl lsp-add public public-lr0
> > +check multinode_nbctl lsp-set-type public-lr0 router
> > +check multinode_nbctl lsp-set-addresses public-lr0 router
> > +check multinode_nbctl lsp-set-options public-lr0 router-port=lr0-public
> > +check multinode_nbctl lrp-set-gateway-chassis lr0-public ovn-gw-1 10
> > +
> > +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
> > +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.120 20.0.0.3
> > +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 10.0.0.0/24
> > +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 20.0.0.0/24
> > +
> > +# Create a logical port pub-p1 and bind it in ovn-chassis-1
> > +check multinode_nbctl lsp-add public public-port1
> > +check multinode_nbctl lsp-set-addresses public-port1 "60:54:00:00:00:03 172.168.0.50"
> > +
> > +m_as ovn-chassis-1 /data/create_fake_vm.sh public-port1 pubp1 60:54:00:00:00:03 172.20.0.50 24 172.20.0.100
> > +
> > +check multinode_nbctl --wait=hv sync
> > +
> > +# First do basic ping tests before deleting the localnet port - ln-public.
> > +# Once the localnet port is deleted from public ls, routing for 172.20.0.0/24
> > +# is centralized on ovn-gw-1.
> > +
> > +# This function checks the North-South traffic.
> > +run_ns_traffic() {
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.110], [ignore], [ignore])
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.120], [ignore], [ignore])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-2], [sw1p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  # Now ping from pubp1 to 172.20.0.100, 172.20.0.110, 172.20.0.120, 10.0.0.3 and 20.0.0.3
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.3 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 20.0.0.3 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +}
> > +
> > +# Test out the N-S traffic.
> > +run_ns_traffic
> > +
> > +# Delete the localnet port by changing the type of ln-public to VIF port.
> > +check multinode_nbctl --wait=hv lsp-set-type ln-public ""
> > +
> > +# cr-port should not be created for public-lr0 since the option
> > +# centralize_routing=true is not yet set for lr0-public.
> > +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > +
> > +# Set the option - centralize_routing now.
> > +check multinode_nbctl --wait=hv set logical_router_port lr0-public options:centralize_routing=true
> > +
> > +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > +m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> > +
> > +# Test out the N-S traffic.
> > +run_ns_traffic
> > +
> > +# Re-add the localnet port
> > +check multinode_nbctl --wait=hv lsp-set-type ln-public localnet
> > +
> > +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > +
> > +# Test out the N-S traffic.
> > +run_ns_traffic
> > +
> > +# Delete the ln-public port this time.
> > +check multinode_nbctl --wait=hv lsp-del ln-public
> > +
> > +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > +m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> > +
> > +# Test out the N-S traffic.
> > +run_ns_traffic
> > +
> >  AT_CLEANUP
> >
> >  AT_SETUP([ovn provider network - always_tunnel])
> > diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
> > index 57f89a7746..649c20f285 100644
> > --- a/tests/ovn-northd.at
> > +++ b/tests/ovn-northd.at
> > @@ -2187,7 +2187,7 @@ match=(inport == "lrp-public" && arp.op == 1 && arp.tpa == 43.43.43.4 && is_chas
> >  action=(eth.dst = eth.src; eth.src = 00:00:00:00:00:02; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 00:00:00:00:00:02; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> >  ])
> >
> > -# xreg0[0..47] isn't used anywhere else.
> > +# xreg0[[0..47]] isn't used anywhere else.
> >  AT_CHECK([ovn-sbctl lflow-list | grep "xreg0\[[0..47\]]" | grep -vE 'lr_in_admission|lr_in_ip_input'], [1], [])
> >
> >  AT_CLEANUP
> > @@ -5503,13 +5503,14 @@ AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | grep "192.168.4.100" | grep "_MC_flo
> >
> >  AS_BOX([Configuring ro1-ls1 router port as a gateway router port])
> >
> > -ovn-nbctl --wait=sb lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> > +check ovn-nbctl  lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> > +check ovn-nbctl --wait=sb lsp-add ls1 ln-ls1 -- lsp-set-type ln-ls1 localnet
> >
> >  ovn-sbctl lflow-list ls1 > ls1_lflows
> >  AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | ovn_strip_lflows], [0], [dnl
> >    table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> >    table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> > -  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01), action=(outport = "ls1-ro1"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01 && is_chassis_resident("cr-ro1-ls1")), action=(outport = "ls1-ro1"; output;)
> >    table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:02), action=(outport = "vm1"; output;)
> >    table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> >    table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:01:01} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > @@ -12475,3 +12476,512 @@ AT_CHECK([ovn-sbctl dump-flows lr | grep lr_in_dnat | ovn_strip_lflows], [0], [d
> >
> >  AT_CLEANUP
> >  ])
> > +
> > +OVN_FOR_EACH_NORTHD_NO_HV([
> > +AT_SETUP([NAT on a provider network with no localnet ports])
> > +AT_KEYWORDS([NAT])
> > +ovn_start
> > +
> > +check ovn-nbctl -- ls-add sw0 -- ls-add sw1
> > +check ovn-nbctl lsp-add sw0 sw0-port1
> > +check ovn-nbctl lr-add lr0
> > +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
> > +check ovn-nbctl lsp-add sw0 sw0-lr0
> > +check ovn-nbctl lsp-set-type sw0-lr0 router
> > +check ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01
> > +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> > +
> > +check ovn-nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:03 20.0.0.1/24
> > +check ovn-nbctl lsp-add sw1 sw1-lr0
> > +check ovn-nbctl lsp-set-type sw1-lr0 router
> > +check ovn-nbctl lsp-set-addresses sw1-lr0 router
> > +check ovn-nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> > +
> > +check ovn-sbctl chassis-add gw1 geneve 127.0.0.1
> > +check ovn-nbctl ls-add public
> > +check ovn-nbctl lsp-add public pub-p1
> > +
> > +# localnet port
> > +check ovn-nbctl lsp-add public ln-public
> > +check ovn-nbctl lsp-set-type ln-public localnet
> > +check ovn-nbctl lsp-set-addresses ln-public unknown
> > +check ovn-nbctl lsp-set-options ln-public network_name=public
> > +
> > +check ovn-nbctl lrp-add lr0 lr0-public 00:00:00:00:ff:02 172.168.0.10/24
> > +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1
> > +
> > +check ovn-nbctl lsp-add public public-lr0
> > +check ovn-nbctl lsp-set-type public-lr0 router
> > +check ovn-nbctl lsp-set-addresses public-lr0 router
> > +check ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public
> > +
> > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
> > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.120 20.0.0.3
> > +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 10.0.0.0/24
> > +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 20.0.0.0/24
> > +
> > +check ovn-nbctl --wait=sb sync
> > +
> > +check_flows_no_cr_port_for_public_lr0() {
> > +  # check that there is no port binding cr-public-lr0
> > +  check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > +
> > +  ovn-sbctl dump-flows lr0 > lr0flows
> > +  ovn-sbctl dump-flows public > publicflows
> > +
> > +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
> > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
> > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +])
> > +
> > +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> > +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-lr0-public")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +}
> > +
> > +check_flows_cr_port_for_public_lr0() {
> > +  # check that there is port binding cr-public-lr0
> > +  check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > +  check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> > +
> > +  ovn-sbctl dump-flows lr0 > lr0flows
> > +  ovn-sbctl dump-flows public > publicflows
> > +
> > +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > +])
> > +
> > +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
> > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
> > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +])
> > +
> > +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> > +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-public-lr0")), action=(outport = "cr-public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-public-lr0")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +}
> > +
> > +# Check that the lflows are as expected when public has localnet port.
> > +check_flows_no_cr_port_for_public_lr0
> > +
> > +# Remove the localnet port from public logical switch.
> > +check ovn-nbctl --wait=sb lsp-set-type ln-public ""
> > +
> > +# Check that the lflows are as expected and there is no cr port
> > +# created for "public-lr0"  when public has no localnet port
> > +# since public doesn't have the option "overlay_provider_network=true"
> > +# set.
> > +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > +
> > +ovn-sbctl dump-flows lr0 > lr0flows
> > +ovn-sbctl dump-flows public > publicflows
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +
> > +
> > +# Set the option "centralize_routing=true" for lr0-public.
> > +check ovn-nbctl --wait=sb set logical_router_port lr0-public options:centralize_routing=true
> > +
> > +# Check that the lflows are as expected and there is cr port created for public-lr0.
> > +check_flows_cr_port_for_public_lr0
> > +
> > +# Set the type of ln-public back to localnet
> > +check ovn-nbctl --wait=sb lsp-set-type ln-public localnet
> > +
> > +# Check that the lflows are as expected when public has localnet port.
> > +check_flows_no_cr_port_for_public_lr0
> > +
> > +# Delete the localnet port
> > +check ovn-nbctl --wait=sb lsp-del ln-public
> > +
> > +# Check that the lflows are as expected when public has no localnet port.
> > +check_flows_cr_port_for_public_lr0
> > +
> > +# Create multiple gateway ports.  chassisresident port should not be
> > +# created for 'public-lr0' even if there is no localnet port on 'public'
> > +# logical switch.
> > +check ovn-nbctl --wait=sb lrp-set-gateway-chassis lr0-sw0 gw1
> > +# check that there is no port binding cr-public-lr0
> > +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > +
> > +ovn-sbctl dump-flows lr0 > lr0flows
> > +ovn-sbctl dump-flows public > publicflows
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +
> > +AT_CLEANUP
> > +])
> > diff --git a/tests/ovn.at b/tests/ovn.at
> > index 582ec56485..c468673f0c 100644
> > --- a/tests/ovn.at
> > +++ b/tests/ovn.at
> > @@ -21227,10 +21227,10 @@ ovn-nbctl lsp-add sw0 rp-sw0 -- set Logical_Switch_Port rp-sw0 \
> >      type=router options:router-port=sw0 \
> >      -- lsp-set-addresses rp-sw0 router
> >
> > -ovn-nbctl lrp-add lr0 sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
> > -    -- lrp-set-gateway-chassis sw1 hv2
> > +ovn-nbctl lrp-add lr0 lr0-sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
> > +    -- lrp-set-gateway-chassis lr0-sw1 hv2
> >  ovn-nbctl lsp-add sw1 rp-sw1 -- set Logical_Switch_Port rp-sw1 \
> > -    type=router options:router-port=sw1 \
> > +    type=router options:router-port=lr0-sw1 \
> >      -- lsp-set-addresses rp-sw1 router
> >
> >  ovn-nbctl lsp-add sw0 sw0-p0 \
> > @@ -21242,6 +21242,8 @@ ovn-nbctl lsp-add sw0 sw0-p1 \
> >  ovn-nbctl lsp-add sw1 sw1-p0 \
> >      -- lsp-set-addresses sw1-p0 unknown
> >
> > +check ovn-nbctl lsp-add sw1 ln-sw1 -- lsp-set-type ln-sw1 localnet
> > +
> >  ovn-nbctl lr-nat-add lr0 snat 172.16.1.1 192.168.1.0/24
> >  ovn-nbctl lr-nat-add lr0 snat 2002::1 2001::/64
> >
> > --
> > 2.45.2
> >
> > _______________________________________________
> > dev mailing list
> > dev@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
Numan Siddique Aug. 7, 2024, 3:35 a.m. UTC | #5 
On Tue, Aug 6, 2024 at 2:50 AM Han Zhou <zhouhan@gmail.com> wrote:
>
> On Mon, Jul 29, 2024 at 7:39 PM <numans@ovn.org> wrote:
> >
> > From: Numan Siddique <numans@ovn.org>
> >
> > Consider a deployment with the below logical resources:
> >
> > 1. A bridged logical switch 'public' with a port - P1 and a localnet
> >    port ln-public.
> > 2. A logical router 'R'
> > 3. Logical switch 'public' connected to R via logical switch/router port
> >    peers (public-R and R-public).
> > 4. R-public is distributed gateway port with its network as 172.16.0.0/24
> > 5. NATs (dnat_and_snat) configured in 'R'.
> > 6. And a few overlay logical switches S1, S2 to R.
> >
> > Any traffic from logical port - P1 of public logical switch destined to
> > S1 or S2's logical ports goes out of the source chassis
> > (where P1 resides) via the localnet port and reaches the gateway chassis
> > which handles the routing.
> >
> > There are couple of traffic flow scenarios which doesn't work if the
> > logical switch 'public' doesn't have a localnet port.
> >
> > 1. Traffic from port - P1 destined to logical switches S1 or S2 gets
> >    dropped in the source chassis.  The packet enters the router R's
> >    pipeline, but it gets dropped in the 'lr_in_admission' stage since
> >    the logical flow to allow traffic destined to the distributed gateway
> >    port MAC is installed only on the gateway chassis.
> >
> > 2. NAT doesn't work as expected.
> >
> > In order to suppose this use case (of a logical switch not having a
> > localnet port, but has a distributed gateway port and NATs), this patch
> > supports the option 'centralize_routing', which can be configured on
> > the distributed gateway port (R-public in the example above).
> > If this option is set, then routing is centralized on the gateway
> > chassis for the traffic destined to the R-public's networks
> > (172.16.0.0/24 for the above example).  Traffic from P1 will be
> > tunnelled to the gateway chassis.
> >
> > ovn-northd creates a chassisresident port (cr-public-R) for the
> > logical switch port - public-R, along with cr-R-public inorder to
> > centralize the traffic.
> >
> Hi Numan, I have two questions here:
>
> > This feature gets enabled for the distributed gateway port R-public if
> >   - The above option is set to true in the R-public's options column.
>
> Why do we need an option to specify this behavior? If the LRP is a
> distributed gateway port and there is no localnet port on the
> connected LS, can't we just do the centralized routing?

That was my initial approach.  But many test cases fail [1] with this approach.
It was hard to fix all these test cases and I was not sure if I'd
break any existing
scenarios.

[1] https://github.com/numansiddique/ovn/actions/runs/10275831079/job/28435316245
testsuite: 75 76 161 162 163 164 165 166 167 168 169 170 183 184 185
186 187 188 271 272 291 292 failed


>
> >   - The logical switch 'public' doesn't have any localnet ports.
> >   - And R-public is the only distributed gateway port of R.
>
> Why do we require that the R-public is the only distributed gateway
> port of R? What if S1 and S2 in this example also connect to R with
> distributed router ports? Can we make it work the same way? Or is it
> just an initial version and to be extended to support any number of
> DGPs in the future?

It's the latter.  There was no use case for us to support this option
with multiple DGPs.
I think it can be extended later if there are use cases.  Do you have
any such use cases in mind ?

Thanks
Numan

>
> Best regards,
> Han
>
> >
> > Distributed NAT (i.e if external_mac and router_port is set) is
> > not supported and instead the router port mac is used for such traffic
> > and centralized on the gateway chassis.
> >
> > Reported-at: https://issues.redhat.com/browse/FDP-364
> > Acked-by: Mark Michelson <mmichels@redhat.com>
> > Signed-off-by: Numan Siddique <numans@ovn.org>
> > ---
> >
> > v1 -> v2
> > -------
> >    * Corrected the NEWS item entry for the new option.
> >    * Rebased and resolved conflicts.
> >
> > Note: This patch is the continuation from this series - https://patchwork.ozlabs.org/project/ovn/patch/20240606214432.168750-1-numans@ovn.org/
> >       Resetted the version number since the new option changed from LSP
> >       to LRP.
> >
> >
> >  NEWS                      |   3 +
> >  controller/physical.c     |   4 +
> >  northd/northd.c           | 257 +++++++++++++++----
> >  northd/northd.h           |   1 +
> >  ovn-nb.xml                |  34 +++
> >  tests/multinode-macros.at |   2 +-
> >  tests/multinode.at        | 177 +++++++++++++
> >  tests/ovn-northd.at       | 516 +++++++++++++++++++++++++++++++++++++-
> >  tests/ovn.at              |   8 +-
> >  9 files changed, 952 insertions(+), 50 deletions(-)
> >
> > diff --git a/NEWS b/NEWS
> > index 87e326f21e..8440a74677 100644
> > --- a/NEWS
> > +++ b/NEWS
> > @@ -47,6 +47,9 @@ Post v24.03.0
> >    - Add support for CT zone limit that can be specified per LR
> >      (options:ct-zone-limit), LS (other_config:ct-zone-limit) or LSP
> >      (options:ct-zone-limit).
> > +  - A new LRP option 'centralize_routing' has been added to a
> > +    distributed gateway port to centralize routing if the logical
> > +    switch of its peer doesn't have a localnet port.
> >
> >  OVN v24.03.0 - 01 Mar 2024
> >  --------------------------
> > diff --git a/controller/physical.c b/controller/physical.c
> > index 3c0200c383..9e04ad5f22 100644
> > --- a/controller/physical.c
> > +++ b/controller/physical.c
> > @@ -1610,6 +1610,10 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> >                                                      ct_zones);
> >              put_zones_ofpacts(&zone_ids, ofpacts_p);
> >
> > +            /* Clear the MFF_INPORT.  Its possible that the same packet may
> > +             * go out from the same tunnel inport. */
> > +            put_load(ofp_to_u16(OFPP_NONE), MFF_IN_PORT, 0, 16, ofpacts_p);
> > +
> >              /* Resubmit to table 41. */
> >              put_resubmit(OFTABLE_CHECK_LOOPBACK, ofpacts_p);
> >          }
> > diff --git a/northd/northd.c b/northd/northd.c
> > index 5c2fd74ff1..4f59d4f1a3 100644
> > --- a/northd/northd.c
> > +++ b/northd/northd.c
> > @@ -2107,6 +2107,55 @@ parse_lsp_addrs(struct ovn_port *op)
> >      }
> >  }
> >
> > +static struct ovn_port *
> > +create_cr_port(struct ovn_port *op, struct hmap *ports,
> > +               struct ovs_list *both_dbs, struct ovs_list *nb_only)
> > +{
> > +    char *redirect_name = ovn_chassis_redirect_name(
> > +        op->nbsp ? op->nbsp->name : op->nbrp->name);
> > +
> > +    struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> > +    if (crp && crp->sb && crp->sb->datapath == op->od->sb) {
> > +        ovn_port_set_nb(crp, NULL, op->nbrp);
> > +        ovs_list_remove(&crp->list);
> > +        ovs_list_push_back(both_dbs, &crp->list);
> > +    } else {
> > +        crp = ovn_port_create(ports, redirect_name,
> > +                              op->nbsp, op->nbrp, NULL);
> > +        ovs_list_push_back(nb_only, &crp->list);
> > +    }
> > +
> > +    crp->primary_port = op;
> > +    op->cr_port = crp;
> > +    crp->od = op->od;
> > +    free(redirect_name);
> > +
> > +    return crp;
> > +}
> > +
> > +/* Returns true if chassis resident port needs to be created for
> > + * op's peer logical switch.  False otherwise.
> > + *
> > + * Chassis resident port needs to be created if the following
> > + * conditionsd are met:
> > + *   - op is a distributed gateway port
> > + *   - op has the option 'centralize_routing' set to true
> > + *   - op is the only distributed gateway port attached to its
> > + *     router
> > + *   - op's peer logical switch has no localnet ports.
> > + */
> > +static bool
> > +peer_needs_cr_port_creation(struct ovn_port *op)
> > +{
> > +    if ((op->nbrp->n_gateway_chassis || op->nbrp->ha_chassis_group)
> > +        && op->od->n_l3dgw_ports == 1 && op->peer && op->peer->nbsp
> > +        && !op->peer->od->n_localnet_ports) {
> > +        return smap_get_bool(&op->nbrp->options, "centralize_routing", false);
> > +    }
> > +
> > +    return false;
> > +}
> > +
> >  static void
> >  join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> >                     struct hmap *ls_datapaths, struct hmap *lr_datapaths,
> > @@ -2214,9 +2263,10 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> >              tag_alloc_add_existing_tags(tag_alloc_table, nbsp);
> >          }
> >      }
> > +
> > +    struct hmapx dgps = HMAPX_INITIALIZER(&dgps);
> >      HMAP_FOR_EACH (od, key_node, lr_datapaths) {
> >          ovs_assert(od->nbr);
> > -        size_t n_allocated_l3dgw_ports = 0;
> >          for (size_t i = 0; i < od->nbr->n_ports; i++) {
> >              const struct nbrec_logical_router_port *nbrp
> >                  = od->nbr->ports[i];
> > @@ -2280,10 +2330,7 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> >                      redirect_type && !strcasecmp(redirect_type, "bridged");
> >              }
> >
> > -            if (op->nbrp->ha_chassis_group ||
> > -                op->nbrp->n_gateway_chassis) {
> > -                /* Additional "derived" ovn_port crp represents the
> > -                 * instance of op on the gateway chassis. */
> > +            if (op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis) {
> >                  const char *gw_chassis = smap_get(&op->od->nbr->options,
> >                                                 "chassis");
> >                  if (gw_chassis) {
> > @@ -2292,34 +2339,9 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> >                      VLOG_WARN_RL(&rl, "Bad configuration: distributed "
> >                                   "gateway port configured on port %s "
> >                                   "on L3 gateway router", nbrp->name);
> > -                    continue;
> > -                }
> > -
> > -                char *redirect_name =
> > -                    ovn_chassis_redirect_name(nbrp->name);
> > -                struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> > -                if (crp && crp->sb && crp->sb->datapath == od->sb) {
> > -                    ovn_port_set_nb(crp, NULL, nbrp);
> > -                    ovs_list_remove(&crp->list);
> > -                    ovs_list_push_back(both, &crp->list);
> >                  } else {
> > -                    crp = ovn_port_create(ports, redirect_name,
> > -                                          NULL, nbrp, NULL);
> > -                    ovs_list_push_back(nb_only, &crp->list);
> > -                }
> > -                crp->primary_port = op;
> > -                op->cr_port = crp;
> > -                crp->od = od;
> > -                free(redirect_name);
> > -
> > -                /* Add to l3dgw_ports in od, for later use during flow
> > -                 * creation. */
> > -                if (od->n_l3dgw_ports == n_allocated_l3dgw_ports) {
> > -                    od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> > -                                                 &n_allocated_l3dgw_ports,
> > -                                                 sizeof *od->l3dgw_ports);
> > +                    hmapx_add(&dgps, op);
> >                  }
> > -                od->l3dgw_ports[od->n_l3dgw_ports++] = op;
> >             }
> >          }
> >      }
> > @@ -2376,12 +2398,6 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> >                          arp_proxy, op->nbsp->name);
> >                  }
> >              }
> > -
> > -            /* Only used for the router type LSP whose peer is l3dgw_port */
> > -            if (op->peer && is_l3dgw_port(op->peer)) {
> > -                op->enable_router_port_acl = smap_get_bool(
> > -                    &op->nbsp->options, "enable_router_port_acl", false);
> > -            }
> >          } else if (op->nbrp && op->nbrp->peer && !is_cr_port(op)) {
> >              struct ovn_port *peer = ovn_port_find(ports, op->nbrp->peer);
> >              if (peer) {
> > @@ -2402,6 +2418,57 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> >          }
> >      }
> >
> > +    struct hmapx_node *hmapx_node;
> > +    HMAPX_FOR_EACH (hmapx_node, &dgps) {
> > +        op = hmapx_node->data;
> > +        od = op->od;
> > +        ovs_assert(op->nbrp);
> > +        ovs_assert(op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis);
> > +
> > +        /* Additional "derived" ovn_port crp represents the instance of op on
> > +         * the gateway chassis. */
> > +        struct ovn_port *crp = create_cr_port(op, ports, both, nb_only);
> > +        ovs_assert(crp);
> > +
> > +        /* Add to l3dgw_ports in od, for later use during flow creation. */
> > +        if (od->n_l3dgw_ports == od->n_allocated_l3dgw_ports) {
> > +            od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> > +                                        &od->n_allocated_l3dgw_ports,
> > +                                        sizeof *od->l3dgw_ports);
> > +        }
> > +        od->l3dgw_ports[od->n_l3dgw_ports++] = op;
> > +
> > +        if (op->peer && op->peer->nbsp) {
> > +            /* Only used for the router type LSP whose peer is l3dgw_port */
> > +            op->peer->enable_router_port_acl = smap_get_bool(
> > +                    &op->peer->nbsp->options, "enable_router_port_acl", false);
> > +        }
> > +    }
> > +
> > +
> > +    /* Create chassisresident port for the distributed gateway port's (DGP)
> > +     * peer if
> > +     *  - DGP's router has only one DGP and
> > +     *  - Its peer is a logical switch port and
> > +     *  - It's peer's logical switch has no localnet ports and
> > +     *  - option 'centralize_routing' is set to true for the DGP.
> > +     *
> > +     * This is required to support
> > +     *   - NAT via geneve (for the overlay provider networks) and
> > +     *   - to centralize routing on the gateway chassis for the traffic
> > +     *     destined to the DGP's networks.
> > +     *
> > +     * Future enhancement: Support 'centralizerouting' for all the DGP's
> > +     * of a logical router.
> > +     * */
> > +    HMAPX_FOR_EACH (hmapx_node, &dgps) {
> > +        op = hmapx_node->data;
> > +        if (peer_needs_cr_port_creation(op)) {
> > +            create_cr_port(op->peer, ports, both, nb_only);
> > +        }
> > +    }
> > +    hmapx_destroy(&dgps);
> > +
> >      /* Wait until all ports have been connected to add to IPAM since
> >       * it relies on proper peers to be set
> >       */
> > @@ -3184,16 +3251,28 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn,
> >               * type "l3gateway". */
> >              if (chassis) {
> >                  sbrec_port_binding_set_type(op->sb, "l3gateway");
> > +            } else if (is_cr_port(op)) {
> > +                sbrec_port_binding_set_type(op->sb, "chassisredirect");
> > +                ovs_assert(op->primary_port->peer);
> > +                ovs_assert(op->primary_port->peer->cr_port);
> > +                ovs_assert(op->primary_port->peer->cr_port->sb);
> > +                sbrec_port_binding_set_ha_chassis_group(
> > +                    op->sb,
> > +                    op->primary_port->peer->cr_port->sb->ha_chassis_group);
> > +
> >              } else {
> >                  sbrec_port_binding_set_type(op->sb, "patch");
> >              }
> >
> >              const char *router_port = smap_get(&op->nbsp->options,
> >                                                 "router-port");
> > -            if (router_port || chassis) {
> > +            if (router_port || chassis || is_cr_port(op)) {
> >                  struct smap new;
> >                  smap_init(&new);
> > -                if (router_port) {
> > +
> > +                if (is_cr_port(op)) {
> > +                    smap_add(&new, "distributed-port", op->nbsp->name);
> > +                } else if (router_port) {
> >                      smap_add(&new, "peer", router_port);
> >                  }
> >                  if (chassis) {
> > @@ -8155,9 +8234,27 @@ build_lswitch_rport_arp_req_flow(
> >      struct lflow_ref *lflow_ref)
> >  {
> >      struct ds match   = DS_EMPTY_INITIALIZER;
> > +    struct ds m       = DS_EMPTY_INITIALIZER;
> >      struct ds actions = DS_EMPTY_INITIALIZER;
> >
> > -    arp_nd_ns_match(ips, addr_family, &match);
> > +    arp_nd_ns_match(ips, addr_family, &m);
> > +    ds_clone(&match, &m);
> > +
> > +    bool has_cr_port = patch_op->cr_port;
> > +
> > +    /* If the patch_op has a chassis resident port, it means
> > +     *    - its peer is a distributed gateway port (DGP) and
> > +     *    - routing is centralized for the DGP's networks on
> > +     *      the configured gateway chassis.
> > +     *
> > +     * If that's the case, make sure that the packets destined to
> > +     * the DGP's MAC are sent to the chassis where the DGP resides.
> > +     * */
> > +
> > +    if (has_cr_port) {
> > +        ds_put_format(&match, " && is_chassis_resident(%s)",
> > +                      patch_op->cr_port->json_key);
> > +    }
> >
> >      /* Send a the packet to the router pipeline.  If the switch has non-router
> >       * ports then flood it there as well.
> > @@ -8179,6 +8276,31 @@ build_lswitch_rport_arp_req_flow(
> >                                  lflow_ref);
> >      }
> >
> > +    if (has_cr_port) {
> > +        ds_clear(&match);
> > +        ds_put_format(&match, "%s && !is_chassis_resident(%s)", ds_cstr(&m),
> > +                      patch_op->cr_port->json_key);
> > +        ds_clear(&actions);
> > +        if (od->n_router_ports != od->nbs->n_ports) {
> > +            ds_put_format(&actions, "clone {outport = %s; output; }; "
> > +                                    "outport = \""MC_FLOOD_L2"\"; output;",
> > +                          patch_op->cr_port->json_key);
> > +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> > +                                    priority, ds_cstr(&match),
> > +                                    ds_cstr(&actions), stage_hint,
> > +                                    lflow_ref);
> > +        } else {
> > +            ds_put_format(&actions, "outport = %s; output;",
> > +                          patch_op->cr_port->json_key);
> > +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> > +                                    priority, ds_cstr(&match),
> > +                                    ds_cstr(&actions),
> > +                                    stage_hint,
> > +                                    lflow_ref);
> > +        }
> > +    }
> > +
> > +    ds_destroy(&m);
> >      ds_destroy(&match);
> >      ds_destroy(&actions);
> >  }
> > @@ -9548,7 +9670,11 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
> >                                  struct ds *actions, struct ds *match)
> >  {
> >      ovs_assert(op->nbsp);
> > -    if (lsp_is_external(op->nbsp)) {
> > +
> > +    /* Note: A switch port can also have a chassis resident derived port.
> > +     * Check if 'op' is a chassis resident dervied port. If so, skip
> > +     * adding unicast lookup flows for this port. */
> > +    if (lsp_is_external(op->nbsp) || is_cr_port(op)) {
> >          return;
> >      }
> >
> > @@ -9566,8 +9692,6 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
> >                             "outport = \""MC_UNKNOWN "\"; output;"
> >                           : "outport = %s; output;")
> >                           : debug_drop_action();
> > -    ds_clear(actions);
> > -    ds_put_format(actions, action, op->json_key);
> >
> >      if (lsp_is_router(op->nbsp) && op->peer && op->peer->nbrp) {
> >          /* For ports connected to logical routers add flows to bypass the
> > @@ -9614,14 +9738,43 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
> >              if (add_chassis_resident_check) {
> >                  ds_put_format(match, " && is_chassis_resident(%s)", json_key);
> >              }
> > +        } else if (op->cr_port) {
> > +            /* If the op has a chassis resident port, it means
> > +             *   - its peer is a distributed gateway port (DGP) and
> > +             *   - routing is centralized for the DGP's networks on
> > +             *     the configured gateway chassis.
> > +             *
> > +             * If that's the case, make sure that the packets destined to
> > +             * the DGP's MAC are sent to the chassis where the DGP resides.
> > +             * */
> > +            ds_clear(actions);
> > +            ds_put_format(actions, action, op->cr_port->json_key);
> > +
> > +            struct ds m = DS_EMPTY_INITIALIZER;
> > +            ds_put_format(&m, "eth.dst == %s && !is_chassis_resident(%s)",
> > +                          op->peer->lrp_networks.ea_s,
> > +                          op->cr_port->json_key);
> > +
> > +            ovn_lflow_add_with_hint(lflows, op->od,
> > +                                    S_SWITCH_IN_L2_LKUP, 50,
> > +                                    ds_cstr(&m), ds_cstr(actions),
> > +                                    &op->nbsp->header_,
> > +                                    op->lflow_ref);
> > +            ds_destroy(&m);
> > +            ds_put_format(match, " && is_chassis_resident(%s)",
> > +                          op->cr_port->json_key);
> >          }
> >
> > +        ds_clear(actions);
> > +        ds_put_format(actions, action, op->json_key);
> >          ovn_lflow_add_with_hint(lflows, op->od,
> >                                  S_SWITCH_IN_L2_LKUP, 50,
> >                                  ds_cstr(match), ds_cstr(actions),
> >                                  &op->nbsp->header_,
> >                                  op->lflow_ref);
> >      } else {
> > +        ds_clear(actions);
> > +        ds_put_format(actions, action, op->json_key);
> >          for (size_t i = 0; i < op->n_lsp_addrs; i++) {
> >              ds_clear(match);
> >              ds_put_format(match, "eth.dst == %s", op->lsp_addrs[i].ea_s);
> > @@ -11725,6 +11878,14 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op,
> >          return;
> >      }
> >
> > +    if (op->peer && op->peer->cr_port) {
> > +        /* We don't add the below flows if the router port's peer has
> > +         * a chassisresident port.  That's because routing is centralized on
> > +         * the gateway chassis for the router port networks/subnets.
> > +         */
> > +        return;
> > +    }
> > +
> >      /* Mac address to use when replying to ARP/NS. */
> >      const char *mac_s = REG_INPORT_ETH_ADDR;
> >      struct eth_addr mac;
> > @@ -15109,6 +15270,16 @@ lrouter_check_nat_entry(const struct ovn_datapath *od,
> >      /* For distributed router NAT, determine whether this NAT rule
> >       * satisfies the conditions for distributed NAT processing. */
> >      *distributed = false;
> > +
> > +    /* NAT cannnot be distributed if the DGP's peer
> > +     * has a chassisresident port (as the routing is centralized
> > +     * on the gateway chassis for the DGP's networks/subnets.)
> > +     */
> > +    struct ovn_port *l3dgw_port = *nat_l3dgw_port;
> > +    if (l3dgw_port && l3dgw_port->peer && l3dgw_port->peer->cr_port) {
> > +        return 0;
> > +    }
> > +
> >      if (od->n_l3dgw_ports && !strcmp(nat->type, "dnat_and_snat") &&
> >          nat->logical_port && nat->external_mac) {
> >          if (eth_addr_from_string(nat->external_mac, mac)) {
> > diff --git a/northd/northd.h b/northd/northd.h
> > index d4a8d75abc..d7c9655916 100644
> > --- a/northd/northd.h
> > +++ b/northd/northd.h
> > @@ -325,6 +325,7 @@ struct ovn_datapath {
> >       * will be NULL. */
> >      struct ovn_port **l3dgw_ports;
> >      size_t n_l3dgw_ports;
> > +    size_t n_allocated_l3dgw_ports;
> >
> >      /* router datapath has a logical port with redirect-type set to bridged. */
> >      bool redirect_bridged;
> > diff --git a/ovn-nb.xml b/ovn-nb.xml
> > index a4362a4ef1..217d1cd1fe 100644
> > --- a/ovn-nb.xml
> > +++ b/ovn-nb.xml
> > @@ -3499,6 +3499,40 @@ or
> >            <ref column="options" key="gateway_mtu"/> option.
> >          </p>
> >        </column>
> > +
> > +      <column name="options" key="centralize_routing"
> > +              type='{"type": "boolean"}'>
> > +        <p>
> > +          This option is applicable only if the router port is a
> > +          distributed gateway port i.e if the <ref table="Logical_Router_Port"
> > +          column="ha_chassis_group"/> column or
> > +          <ref table="Logical_Router_Port" column="gateway_chassis"/>
> > +          is set.
> > +        </p>
> > +
> > +        <p>
> > +          If set to <code>true</code>, routing for the router port's
> > +          networks (set in the column <ref table="Logical_Router_Port"
> > +          column="networks"/>) is centralized on the gateway chassis
> > +          which claims this distributed gateway port.
> > +        </p>
> > +
> > +        <p>
> > +          Additionally for this option to take effect, below conditions
> > +          must be met:
> > +        </p>
> > +
> > +        <ul>
> > +          <li>
> > +            The Logical router has only one distributed gateway port.
> > +          </li>
> > +
> > +          <li>
> > +            The router port's peer logical switch has no localnet ports.
> > +          </li>
> > +
> > +        </ul>
> > +      </column>
> >      </group>
> >
> >      <group title="Attachment">
> > diff --git a/tests/multinode-macros.at b/tests/multinode-macros.at
> > index 786e564860..757917626c 100644
> > --- a/tests/multinode-macros.at
> > +++ b/tests/multinode-macros.at
> > @@ -92,7 +92,7 @@ m_count_rows() {
> >  m_check_row_count() {
> >      local db=$(parse_db $1) table=$(parse_table $1); shift
> >      local count=$1; shift
> > -    local found=$(m_count_rows $c $db:$table "$@")
> > +    local found=$(m_count_rows $db:$table "$@")
> >      echo
> >      echo "Checking for $count rows in $db $table${1+ with $*}... found $found"
> >      if test "$count" != "$found"; then
> > diff --git a/tests/multinode.at b/tests/multinode.at
> > index a7231130ac..a725414165 100644
> > --- a/tests/multinode.at
> > +++ b/tests/multinode.at
> > @@ -1033,6 +1033,183 @@ M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [sh -c 'dd bs=512 count=2 if=/dev/uran
> >  done
> >  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ip route get 10.0.0.1 dev sw0p1 | grep -q 'mtu 942'])
> >
> > +# Reset back to geneve tunnels
> > +for c in ovn-chassis-1 ovn-chassis-2 ovn-gw-1
> > +do
> > +    m_as $c ovs-vsctl set open . external-ids:ovn-encap-type=geneve
> > +done
> > +
> > +AT_CLEANUP
> > +
> > +AT_SETUP([ovn multinode NAT on a provider network with no localnet ports])
> > +
> > +# Check that ovn-fake-multinode setup is up and running
> > +check_fake_multinode_setup
> > +
> > +# Delete the multinode NB and OVS resources before starting the test.
> > +cleanup_multinode_resources
> > +
> > +check multinode_nbctl ls-add sw0
> > +check multinode_nbctl lsp-add sw0 sw0-port1
> > +check multinode_nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03 10.0.0.3 1000::3"
> > +check multinode_nbctl lsp-add sw0 sw0-port2
> > +check multinode_nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:04 10.0.0.4 1000::4"
> > +
> > +m_as ovn-chassis-1 /data/create_fake_vm.sh sw0-port1 sw0p1 50:54:00:00:00:03 10.0.0.3 24 10.0.0.1 1000::3/64 1000::a
> > +m_as ovn-chassis-2 /data/create_fake_vm.sh sw0-port2 sw0p2 50:54:00:00:00:04 10.0.0.4 24 10.0.0.1 1000::4/64 1000::a
> > +
> > +m_wait_for_ports_up
> > +
> > +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.4 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +# Create the second logical switch with one port
> > +check multinode_nbctl ls-add sw1
> > +check multinode_nbctl lsp-add sw1 sw1-port1
> > +check multinode_nbctl lsp-set-addresses sw1-port1 "40:54:00:00:00:03 20.0.0.3 2000::3"
> > +
> > +# Create a logical router and attach both logical switches
> > +check multinode_nbctl lr-add lr0
> > +check multinode_nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 1000::a/64
> > +check multinode_nbctl lsp-add sw0 sw0-lr0
> > +check multinode_nbctl lsp-set-type sw0-lr0 router
> > +check multinode_nbctl lsp-set-addresses sw0-lr0 router
> > +check multinode_nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> > +
> > +check multinode_nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:02 20.0.0.1/24 2000::a/64
> > +check multinode_nbctl lsp-add sw1 sw1-lr0
> > +check multinode_nbctl lsp-set-type sw1-lr0 router
> > +check multinode_nbctl lsp-set-addresses sw1-lr0 router
> > +check multinode_nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> > +
> > +m_as ovn-chassis-2 /data/create_fake_vm.sh sw1-port1 sw1p1 40:54:00:00:00:03 20.0.0.3 24 20.0.0.1 2000::3/64 2000::a
> > +
> > +# create exteranl connection for N/S traffic
> > +check multinode_nbctl ls-add public
> > +check multinode_nbctl lsp-add public ln-public
> > +check multinode_nbctl lsp-set-type ln-public localnet
> > +check multinode_nbctl lsp-set-addresses ln-public unknown
> > +check multinode_nbctl lsp-set-options ln-public network_name=public
> > +
> > +check multinode_nbctl lrp-add lr0 lr0-public 00:11:22:00:ff:01 172.20.0.100/24
> > +check multinode_nbctl lsp-add public public-lr0
> > +check multinode_nbctl lsp-set-type public-lr0 router
> > +check multinode_nbctl lsp-set-addresses public-lr0 router
> > +check multinode_nbctl lsp-set-options public-lr0 router-port=lr0-public
> > +check multinode_nbctl lrp-set-gateway-chassis lr0-public ovn-gw-1 10
> > +
> > +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
> > +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.120 20.0.0.3
> > +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 10.0.0.0/24
> > +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 20.0.0.0/24
> > +
> > +# Create a logical port pub-p1 and bind it in ovn-chassis-1
> > +check multinode_nbctl lsp-add public public-port1
> > +check multinode_nbctl lsp-set-addresses public-port1 "60:54:00:00:00:03 172.168.0.50"
> > +
> > +m_as ovn-chassis-1 /data/create_fake_vm.sh public-port1 pubp1 60:54:00:00:00:03 172.20.0.50 24 172.20.0.100
> > +
> > +check multinode_nbctl --wait=hv sync
> > +
> > +# First do basic ping tests before deleting the localnet port - ln-public.
> > +# Once the localnet port is deleted from public ls, routing for 172.20.0.0/24
> > +# is centralized on ovn-gw-1.
> > +
> > +# This function checks the North-South traffic.
> > +run_ns_traffic() {
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.110], [ignore], [ignore])
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.120], [ignore], [ignore])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-2], [sw1p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  # Now ping from pubp1 to 172.20.0.100, 172.20.0.110, 172.20.0.120, 10.0.0.3 and 20.0.0.3
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.3 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 20.0.0.3 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +}
> > +
> > +# Test out the N-S traffic.
> > +run_ns_traffic
> > +
> > +# Delete the localnet port by changing the type of ln-public to VIF port.
> > +check multinode_nbctl --wait=hv lsp-set-type ln-public ""
> > +
> > +# cr-port should not be created for public-lr0 since the option
> > +# centralize_routing=true is not yet set for lr0-public.
> > +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > +
> > +# Set the option - centralize_routing now.
> > +check multinode_nbctl --wait=hv set logical_router_port lr0-public options:centralize_routing=true
> > +
> > +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > +m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> > +
> > +# Test out the N-S traffic.
> > +run_ns_traffic
> > +
> > +# Re-add the localnet port
> > +check multinode_nbctl --wait=hv lsp-set-type ln-public localnet
> > +
> > +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > +
> > +# Test out the N-S traffic.
> > +run_ns_traffic
> > +
> > +# Delete the ln-public port this time.
> > +check multinode_nbctl --wait=hv lsp-del ln-public
> > +
> > +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > +m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> > +
> > +# Test out the N-S traffic.
> > +run_ns_traffic
> > +
> >  AT_CLEANUP
> >
> >  AT_SETUP([ovn provider network - always_tunnel])
> > diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
> > index 57f89a7746..649c20f285 100644
> > --- a/tests/ovn-northd.at
> > +++ b/tests/ovn-northd.at
> > @@ -2187,7 +2187,7 @@ match=(inport == "lrp-public" && arp.op == 1 && arp.tpa == 43.43.43.4 && is_chas
> >  action=(eth.dst = eth.src; eth.src = 00:00:00:00:00:02; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 00:00:00:00:00:02; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> >  ])
> >
> > -# xreg0[0..47] isn't used anywhere else.
> > +# xreg0[[0..47]] isn't used anywhere else.
> >  AT_CHECK([ovn-sbctl lflow-list | grep "xreg0\[[0..47\]]" | grep -vE 'lr_in_admission|lr_in_ip_input'], [1], [])
> >
> >  AT_CLEANUP
> > @@ -5503,13 +5503,14 @@ AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | grep "192.168.4.100" | grep "_MC_flo
> >
> >  AS_BOX([Configuring ro1-ls1 router port as a gateway router port])
> >
> > -ovn-nbctl --wait=sb lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> > +check ovn-nbctl  lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> > +check ovn-nbctl --wait=sb lsp-add ls1 ln-ls1 -- lsp-set-type ln-ls1 localnet
> >
> >  ovn-sbctl lflow-list ls1 > ls1_lflows
> >  AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | ovn_strip_lflows], [0], [dnl
> >    table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> >    table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> > -  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01), action=(outport = "ls1-ro1"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01 && is_chassis_resident("cr-ro1-ls1")), action=(outport = "ls1-ro1"; output;)
> >    table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:02), action=(outport = "vm1"; output;)
> >    table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> >    table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:01:01} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > @@ -12475,3 +12476,512 @@ AT_CHECK([ovn-sbctl dump-flows lr | grep lr_in_dnat | ovn_strip_lflows], [0], [d
> >
> >  AT_CLEANUP
> >  ])
> > +
> > +OVN_FOR_EACH_NORTHD_NO_HV([
> > +AT_SETUP([NAT on a provider network with no localnet ports])
> > +AT_KEYWORDS([NAT])
> > +ovn_start
> > +
> > +check ovn-nbctl -- ls-add sw0 -- ls-add sw1
> > +check ovn-nbctl lsp-add sw0 sw0-port1
> > +check ovn-nbctl lr-add lr0
> > +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
> > +check ovn-nbctl lsp-add sw0 sw0-lr0
> > +check ovn-nbctl lsp-set-type sw0-lr0 router
> > +check ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01
> > +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> > +
> > +check ovn-nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:03 20.0.0.1/24
> > +check ovn-nbctl lsp-add sw1 sw1-lr0
> > +check ovn-nbctl lsp-set-type sw1-lr0 router
> > +check ovn-nbctl lsp-set-addresses sw1-lr0 router
> > +check ovn-nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> > +
> > +check ovn-sbctl chassis-add gw1 geneve 127.0.0.1
> > +check ovn-nbctl ls-add public
> > +check ovn-nbctl lsp-add public pub-p1
> > +
> > +# localnet port
> > +check ovn-nbctl lsp-add public ln-public
> > +check ovn-nbctl lsp-set-type ln-public localnet
> > +check ovn-nbctl lsp-set-addresses ln-public unknown
> > +check ovn-nbctl lsp-set-options ln-public network_name=public
> > +
> > +check ovn-nbctl lrp-add lr0 lr0-public 00:00:00:00:ff:02 172.168.0.10/24
> > +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1
> > +
> > +check ovn-nbctl lsp-add public public-lr0
> > +check ovn-nbctl lsp-set-type public-lr0 router
> > +check ovn-nbctl lsp-set-addresses public-lr0 router
> > +check ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public
> > +
> > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
> > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.120 20.0.0.3
> > +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 10.0.0.0/24
> > +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 20.0.0.0/24
> > +
> > +check ovn-nbctl --wait=sb sync
> > +
> > +check_flows_no_cr_port_for_public_lr0() {
> > +  # check that there is no port binding cr-public-lr0
> > +  check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > +
> > +  ovn-sbctl dump-flows lr0 > lr0flows
> > +  ovn-sbctl dump-flows public > publicflows
> > +
> > +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
> > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
> > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +])
> > +
> > +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> > +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-lr0-public")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +}
> > +
> > +check_flows_cr_port_for_public_lr0() {
> > +  # check that there is port binding cr-public-lr0
> > +  check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > +  check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> > +
> > +  ovn-sbctl dump-flows lr0 > lr0flows
> > +  ovn-sbctl dump-flows public > publicflows
> > +
> > +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> > +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
> > +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > +])
> > +
> > +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
> > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
> > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > +])
> > +
> > +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +])
> > +
> > +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> > +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-public-lr0")), action=(outport = "cr-public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-public-lr0")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +}
> > +
> > +# Check that the lflows are as expected when public has localnet port.
> > +check_flows_no_cr_port_for_public_lr0
> > +
> > +# Remove the localnet port from public logical switch.
> > +check ovn-nbctl --wait=sb lsp-set-type ln-public ""
> > +
> > +# Check that the lflows are as expected and there is no cr port
> > +# created for "public-lr0"  when public has no localnet port
> > +# since public doesn't have the option "overlay_provider_network=true"
> > +# set.
> > +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > +
> > +ovn-sbctl dump-flows lr0 > lr0flows
> > +ovn-sbctl dump-flows public > publicflows
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +
> > +
> > +# Set the option "centralize_routing=true" for lr0-public.
> > +check ovn-nbctl --wait=sb set logical_router_port lr0-public options:centralize_routing=true
> > +
> > +# Check that the lflows are as expected and there is cr port created for public-lr0.
> > +check_flows_cr_port_for_public_lr0
> > +
> > +# Set the type of ln-public back to localnet
> > +check ovn-nbctl --wait=sb lsp-set-type ln-public localnet
> > +
> > +# Check that the lflows are as expected when public has localnet port.
> > +check_flows_no_cr_port_for_public_lr0
> > +
> > +# Delete the localnet port
> > +check ovn-nbctl --wait=sb lsp-del ln-public
> > +
> > +# Check that the lflows are as expected when public has no localnet port.
> > +check_flows_cr_port_for_public_lr0
> > +
> > +# Create multiple gateway ports.  chassisresident port should not be
> > +# created for 'public-lr0' even if there is no localnet port on 'public'
> > +# logical switch.
> > +check ovn-nbctl --wait=sb lrp-set-gateway-chassis lr0-sw0 gw1
> > +# check that there is no port binding cr-public-lr0
> > +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > +
> > +ovn-sbctl dump-flows lr0 > lr0flows
> > +ovn-sbctl dump-flows public > publicflows
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > +])
> > +
> > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > +])
> > +
> > +AT_CLEANUP
> > +])
> > diff --git a/tests/ovn.at b/tests/ovn.at
> > index 582ec56485..c468673f0c 100644
> > --- a/tests/ovn.at
> > +++ b/tests/ovn.at
> > @@ -21227,10 +21227,10 @@ ovn-nbctl lsp-add sw0 rp-sw0 -- set Logical_Switch_Port rp-sw0 \
> >      type=router options:router-port=sw0 \
> >      -- lsp-set-addresses rp-sw0 router
> >
> > -ovn-nbctl lrp-add lr0 sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
> > -    -- lrp-set-gateway-chassis sw1 hv2
> > +ovn-nbctl lrp-add lr0 lr0-sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
> > +    -- lrp-set-gateway-chassis lr0-sw1 hv2
> >  ovn-nbctl lsp-add sw1 rp-sw1 -- set Logical_Switch_Port rp-sw1 \
> > -    type=router options:router-port=sw1 \
> > +    type=router options:router-port=lr0-sw1 \
> >      -- lsp-set-addresses rp-sw1 router
> >
> >  ovn-nbctl lsp-add sw0 sw0-p0 \
> > @@ -21242,6 +21242,8 @@ ovn-nbctl lsp-add sw0 sw0-p1 \
> >  ovn-nbctl lsp-add sw1 sw1-p0 \
> >      -- lsp-set-addresses sw1-p0 unknown
> >
> > +check ovn-nbctl lsp-add sw1 ln-sw1 -- lsp-set-type ln-sw1 localnet
> > +
> >  ovn-nbctl lr-nat-add lr0 snat 172.16.1.1 192.168.1.0/24
> >  ovn-nbctl lr-nat-add lr0 snat 2002::1 2001::/64
> >
> > --
> > 2.45.2
> >
> > _______________________________________________
> > dev mailing list
> > dev@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Han Zhou Aug. 7, 2024, 7:11 a.m. UTC | #6 
On Tue, Aug 6, 2024 at 8:35 PM Numan Siddique <numans@ovn.org> wrote:
>
> On Tue, Aug 6, 2024 at 2:50 AM Han Zhou <zhouhan@gmail.com> wrote:
> >
> > On Mon, Jul 29, 2024 at 7:39 PM <numans@ovn.org> wrote:
> > >
> > > From: Numan Siddique <numans@ovn.org>
> > >
> > > Consider a deployment with the below logical resources:
> > >
> > > 1. A bridged logical switch 'public' with a port - P1 and a localnet
> > >    port ln-public.
> > > 2. A logical router 'R'
> > > 3. Logical switch 'public' connected to R via logical switch/router port
> > >    peers (public-R and R-public).
> > > 4. R-public is distributed gateway port with its network as 172.16.0.0/24
> > > 5. NATs (dnat_and_snat) configured in 'R'.
> > > 6. And a few overlay logical switches S1, S2 to R.
> > >
> > > Any traffic from logical port - P1 of public logical switch destined to
> > > S1 or S2's logical ports goes out of the source chassis
> > > (where P1 resides) via the localnet port and reaches the gateway chassis
> > > which handles the routing.
> > >
> > > There are couple of traffic flow scenarios which doesn't work if the
> > > logical switch 'public' doesn't have a localnet port.
> > >
> > > 1. Traffic from port - P1 destined to logical switches S1 or S2 gets
> > >    dropped in the source chassis.  The packet enters the router R's
> > >    pipeline, but it gets dropped in the 'lr_in_admission' stage since
> > >    the logical flow to allow traffic destined to the distributed gateway
> > >    port MAC is installed only on the gateway chassis.
> > >
> > > 2. NAT doesn't work as expected.
> > >
> > > In order to suppose this use case (of a logical switch not having a
> > > localnet port, but has a distributed gateway port and NATs), this patch
> > > supports the option 'centralize_routing', which can be configured on
> > > the distributed gateway port (R-public in the example above).
> > > If this option is set, then routing is centralized on the gateway
> > > chassis for the traffic destined to the R-public's networks
> > > (172.16.0.0/24 for the above example).  Traffic from P1 will be
> > > tunnelled to the gateway chassis.
> > >
> > > ovn-northd creates a chassisresident port (cr-public-R) for the
> > > logical switch port - public-R, along with cr-R-public inorder to
> > > centralize the traffic.
> > >
> > Hi Numan, I have two questions here:
> >
> > > This feature gets enabled for the distributed gateway port R-public if
> > >   - The above option is set to true in the R-public's options column.
> >
> > Why do we need an option to specify this behavior? If the LRP is a
> > distributed gateway port and there is no localnet port on the
> > connected LS, can't we just do the centralized routing?
>
> That was my initial approach.  But many test cases fail [1] with this approach.
> It was hard to fix all these test cases and I was not sure if I'd
> break any existing
> scenarios.
>
> [1] https://github.com/numansiddique/ovn/actions/runs/10275831079/job/28435316245
> testsuite: 75 76 161 162 163 164 165 166 167 168 169 170 183 184 185
> 186 187 188 271 272 291 292 failed
>

I am a little concerned about this. The original behavior is like a
bug, because a LSP on the external overlay network is supposed to be
able to communicate with the LSPs in the internal overlay networks. So
I understand that this patch is trying to fix that. However, the fix
would break some scenarios and it is not very clear why they break
what scenarios would break. Because of this you added a new option to
workaround the problem, because the test cases that didn't enable this
option would still pass for sure. However, it also means that when
this option is enabled, we are not very sure what other scenarios
would break. Would it be better to examine the failed cases to
understand what is really broken?

>
> >
> > >   - The logical switch 'public' doesn't have any localnet ports.
> > >   - And R-public is the only distributed gateway port of R.
> >
> > Why do we require that the R-public is the only distributed gateway
> > port of R? What if S1 and S2 in this example also connect to R with
> > distributed router ports? Can we make it work the same way? Or is it
> > just an initial version and to be extended to support any number of
> > DGPs in the future?
>
> It's the latter.  There was no use case for us to support this option
> with multiple DGPs.
> I think it can be extended later if there are use cases.  Do you have
> any such use cases in mind ?

I don't have a concrete use case in mind. Just thought about the
ovn-k8s default mode (non-IC) that each node uses DGP on the cluster
LR, but there is no such provider/public network connected to the LR.
So I agree that we can add it later when needed.

Thanks,
Han

>
> Thanks
> Numan
>
> >
> > Best regards,
> > Han
> >
> > >
> > > Distributed NAT (i.e if external_mac and router_port is set) is
> > > not supported and instead the router port mac is used for such traffic
> > > and centralized on the gateway chassis.
> > >
> > > Reported-at: https://issues.redhat.com/browse/FDP-364
> > > Acked-by: Mark Michelson <mmichels@redhat.com>
> > > Signed-off-by: Numan Siddique <numans@ovn.org>
> > > ---
> > >
> > > v1 -> v2
> > > -------
> > >    * Corrected the NEWS item entry for the new option.
> > >    * Rebased and resolved conflicts.
> > >
> > > Note: This patch is the continuation from this series - https://patchwork.ozlabs.org/project/ovn/patch/20240606214432.168750-1-numans@ovn.org/
> > >       Resetted the version number since the new option changed from LSP
> > >       to LRP.
> > >
> > >
> > >  NEWS                      |   3 +
> > >  controller/physical.c     |   4 +
> > >  northd/northd.c           | 257 +++++++++++++++----
> > >  northd/northd.h           |   1 +
> > >  ovn-nb.xml                |  34 +++
> > >  tests/multinode-macros.at |   2 +-
> > >  tests/multinode.at        | 177 +++++++++++++
> > >  tests/ovn-northd.at       | 516 +++++++++++++++++++++++++++++++++++++-
> > >  tests/ovn.at              |   8 +-
> > >  9 files changed, 952 insertions(+), 50 deletions(-)
> > >
> > > diff --git a/NEWS b/NEWS
> > > index 87e326f21e..8440a74677 100644
> > > --- a/NEWS
> > > +++ b/NEWS
> > > @@ -47,6 +47,9 @@ Post v24.03.0
> > >    - Add support for CT zone limit that can be specified per LR
> > >      (options:ct-zone-limit), LS (other_config:ct-zone-limit) or LSP
> > >      (options:ct-zone-limit).
> > > +  - A new LRP option 'centralize_routing' has been added to a
> > > +    distributed gateway port to centralize routing if the logical
> > > +    switch of its peer doesn't have a localnet port.
> > >
> > >  OVN v24.03.0 - 01 Mar 2024
> > >  --------------------------
> > > diff --git a/controller/physical.c b/controller/physical.c
> > > index 3c0200c383..9e04ad5f22 100644
> > > --- a/controller/physical.c
> > > +++ b/controller/physical.c
> > > @@ -1610,6 +1610,10 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> > >                                                      ct_zones);
> > >              put_zones_ofpacts(&zone_ids, ofpacts_p);
> > >
> > > +            /* Clear the MFF_INPORT.  Its possible that the same packet may
> > > +             * go out from the same tunnel inport. */
> > > +            put_load(ofp_to_u16(OFPP_NONE), MFF_IN_PORT, 0, 16, ofpacts_p);
> > > +
> > >              /* Resubmit to table 41. */
> > >              put_resubmit(OFTABLE_CHECK_LOOPBACK, ofpacts_p);
> > >          }
> > > diff --git a/northd/northd.c b/northd/northd.c
> > > index 5c2fd74ff1..4f59d4f1a3 100644
> > > --- a/northd/northd.c
> > > +++ b/northd/northd.c
> > > @@ -2107,6 +2107,55 @@ parse_lsp_addrs(struct ovn_port *op)
> > >      }
> > >  }
> > >
> > > +static struct ovn_port *
> > > +create_cr_port(struct ovn_port *op, struct hmap *ports,
> > > +               struct ovs_list *both_dbs, struct ovs_list *nb_only)
> > > +{
> > > +    char *redirect_name = ovn_chassis_redirect_name(
> > > +        op->nbsp ? op->nbsp->name : op->nbrp->name);
> > > +
> > > +    struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> > > +    if (crp && crp->sb && crp->sb->datapath == op->od->sb) {
> > > +        ovn_port_set_nb(crp, NULL, op->nbrp);
> > > +        ovs_list_remove(&crp->list);
> > > +        ovs_list_push_back(both_dbs, &crp->list);
> > > +    } else {
> > > +        crp = ovn_port_create(ports, redirect_name,
> > > +                              op->nbsp, op->nbrp, NULL);
> > > +        ovs_list_push_back(nb_only, &crp->list);
> > > +    }
> > > +
> > > +    crp->primary_port = op;
> > > +    op->cr_port = crp;
> > > +    crp->od = op->od;
> > > +    free(redirect_name);
> > > +
> > > +    return crp;
> > > +}
> > > +
> > > +/* Returns true if chassis resident port needs to be created for
> > > + * op's peer logical switch.  False otherwise.
> > > + *
> > > + * Chassis resident port needs to be created if the following
> > > + * conditionsd are met:
> > > + *   - op is a distributed gateway port
> > > + *   - op has the option 'centralize_routing' set to true
> > > + *   - op is the only distributed gateway port attached to its
> > > + *     router
> > > + *   - op's peer logical switch has no localnet ports.
> > > + */
> > > +static bool
> > > +peer_needs_cr_port_creation(struct ovn_port *op)
> > > +{
> > > +    if ((op->nbrp->n_gateway_chassis || op->nbrp->ha_chassis_group)
> > > +        && op->od->n_l3dgw_ports == 1 && op->peer && op->peer->nbsp
> > > +        && !op->peer->od->n_localnet_ports) {
> > > +        return smap_get_bool(&op->nbrp->options, "centralize_routing", false);
> > > +    }
> > > +
> > > +    return false;
> > > +}
> > > +
> > >  static void
> > >  join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> > >                     struct hmap *ls_datapaths, struct hmap *lr_datapaths,
> > > @@ -2214,9 +2263,10 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> > >              tag_alloc_add_existing_tags(tag_alloc_table, nbsp);
> > >          }
> > >      }
> > > +
> > > +    struct hmapx dgps = HMAPX_INITIALIZER(&dgps);
> > >      HMAP_FOR_EACH (od, key_node, lr_datapaths) {
> > >          ovs_assert(od->nbr);
> > > -        size_t n_allocated_l3dgw_ports = 0;
> > >          for (size_t i = 0; i < od->nbr->n_ports; i++) {
> > >              const struct nbrec_logical_router_port *nbrp
> > >                  = od->nbr->ports[i];
> > > @@ -2280,10 +2330,7 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> > >                      redirect_type && !strcasecmp(redirect_type, "bridged");
> > >              }
> > >
> > > -            if (op->nbrp->ha_chassis_group ||
> > > -                op->nbrp->n_gateway_chassis) {
> > > -                /* Additional "derived" ovn_port crp represents the
> > > -                 * instance of op on the gateway chassis. */
> > > +            if (op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis) {
> > >                  const char *gw_chassis = smap_get(&op->od->nbr->options,
> > >                                                 "chassis");
> > >                  if (gw_chassis) {
> > > @@ -2292,34 +2339,9 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> > >                      VLOG_WARN_RL(&rl, "Bad configuration: distributed "
> > >                                   "gateway port configured on port %s "
> > >                                   "on L3 gateway router", nbrp->name);
> > > -                    continue;
> > > -                }
> > > -
> > > -                char *redirect_name =
> > > -                    ovn_chassis_redirect_name(nbrp->name);
> > > -                struct ovn_port *crp = ovn_port_find(ports, redirect_name);
> > > -                if (crp && crp->sb && crp->sb->datapath == od->sb) {
> > > -                    ovn_port_set_nb(crp, NULL, nbrp);
> > > -                    ovs_list_remove(&crp->list);
> > > -                    ovs_list_push_back(both, &crp->list);
> > >                  } else {
> > > -                    crp = ovn_port_create(ports, redirect_name,
> > > -                                          NULL, nbrp, NULL);
> > > -                    ovs_list_push_back(nb_only, &crp->list);
> > > -                }
> > > -                crp->primary_port = op;
> > > -                op->cr_port = crp;
> > > -                crp->od = od;
> > > -                free(redirect_name);
> > > -
> > > -                /* Add to l3dgw_ports in od, for later use during flow
> > > -                 * creation. */
> > > -                if (od->n_l3dgw_ports == n_allocated_l3dgw_ports) {
> > > -                    od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> > > -                                                 &n_allocated_l3dgw_ports,
> > > -                                                 sizeof *od->l3dgw_ports);
> > > +                    hmapx_add(&dgps, op);
> > >                  }
> > > -                od->l3dgw_ports[od->n_l3dgw_ports++] = op;
> > >             }
> > >          }
> > >      }
> > > @@ -2376,12 +2398,6 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> > >                          arp_proxy, op->nbsp->name);
> > >                  }
> > >              }
> > > -
> > > -            /* Only used for the router type LSP whose peer is l3dgw_port */
> > > -            if (op->peer && is_l3dgw_port(op->peer)) {
> > > -                op->enable_router_port_acl = smap_get_bool(
> > > -                    &op->nbsp->options, "enable_router_port_acl", false);
> > > -            }
> > >          } else if (op->nbrp && op->nbrp->peer && !is_cr_port(op)) {
> > >              struct ovn_port *peer = ovn_port_find(ports, op->nbrp->peer);
> > >              if (peer) {
> > > @@ -2402,6 +2418,57 @@ join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
> > >          }
> > >      }
> > >
> > > +    struct hmapx_node *hmapx_node;
> > > +    HMAPX_FOR_EACH (hmapx_node, &dgps) {
> > > +        op = hmapx_node->data;
> > > +        od = op->od;
> > > +        ovs_assert(op->nbrp);
> > > +        ovs_assert(op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis);
> > > +
> > > +        /* Additional "derived" ovn_port crp represents the instance of op on
> > > +         * the gateway chassis. */
> > > +        struct ovn_port *crp = create_cr_port(op, ports, both, nb_only);
> > > +        ovs_assert(crp);
> > > +
> > > +        /* Add to l3dgw_ports in od, for later use during flow creation. */
> > > +        if (od->n_l3dgw_ports == od->n_allocated_l3dgw_ports) {
> > > +            od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
> > > +                                        &od->n_allocated_l3dgw_ports,
> > > +                                        sizeof *od->l3dgw_ports);
> > > +        }
> > > +        od->l3dgw_ports[od->n_l3dgw_ports++] = op;
> > > +
> > > +        if (op->peer && op->peer->nbsp) {
> > > +            /* Only used for the router type LSP whose peer is l3dgw_port */
> > > +            op->peer->enable_router_port_acl = smap_get_bool(
> > > +                    &op->peer->nbsp->options, "enable_router_port_acl", false);
> > > +        }
> > > +    }
> > > +
> > > +
> > > +    /* Create chassisresident port for the distributed gateway port's (DGP)
> > > +     * peer if
> > > +     *  - DGP's router has only one DGP and
> > > +     *  - Its peer is a logical switch port and
> > > +     *  - It's peer's logical switch has no localnet ports and
> > > +     *  - option 'centralize_routing' is set to true for the DGP.
> > > +     *
> > > +     * This is required to support
> > > +     *   - NAT via geneve (for the overlay provider networks) and
> > > +     *   - to centralize routing on the gateway chassis for the traffic
> > > +     *     destined to the DGP's networks.
> > > +     *
> > > +     * Future enhancement: Support 'centralizerouting' for all the DGP's
> > > +     * of a logical router.
> > > +     * */
> > > +    HMAPX_FOR_EACH (hmapx_node, &dgps) {
> > > +        op = hmapx_node->data;
> > > +        if (peer_needs_cr_port_creation(op)) {
> > > +            create_cr_port(op->peer, ports, both, nb_only);
> > > +        }
> > > +    }
> > > +    hmapx_destroy(&dgps);
> > > +
> > >      /* Wait until all ports have been connected to add to IPAM since
> > >       * it relies on proper peers to be set
> > >       */
> > > @@ -3184,16 +3251,28 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn,
> > >               * type "l3gateway". */
> > >              if (chassis) {
> > >                  sbrec_port_binding_set_type(op->sb, "l3gateway");
> > > +            } else if (is_cr_port(op)) {
> > > +                sbrec_port_binding_set_type(op->sb, "chassisredirect");
> > > +                ovs_assert(op->primary_port->peer);
> > > +                ovs_assert(op->primary_port->peer->cr_port);
> > > +                ovs_assert(op->primary_port->peer->cr_port->sb);
> > > +                sbrec_port_binding_set_ha_chassis_group(
> > > +                    op->sb,
> > > +                    op->primary_port->peer->cr_port->sb->ha_chassis_group);
> > > +
> > >              } else {
> > >                  sbrec_port_binding_set_type(op->sb, "patch");
> > >              }
> > >
> > >              const char *router_port = smap_get(&op->nbsp->options,
> > >                                                 "router-port");
> > > -            if (router_port || chassis) {
> > > +            if (router_port || chassis || is_cr_port(op)) {
> > >                  struct smap new;
> > >                  smap_init(&new);
> > > -                if (router_port) {
> > > +
> > > +                if (is_cr_port(op)) {
> > > +                    smap_add(&new, "distributed-port", op->nbsp->name);
> > > +                } else if (router_port) {
> > >                      smap_add(&new, "peer", router_port);
> > >                  }
> > >                  if (chassis) {
> > > @@ -8155,9 +8234,27 @@ build_lswitch_rport_arp_req_flow(
> > >      struct lflow_ref *lflow_ref)
> > >  {
> > >      struct ds match   = DS_EMPTY_INITIALIZER;
> > > +    struct ds m       = DS_EMPTY_INITIALIZER;
> > >      struct ds actions = DS_EMPTY_INITIALIZER;
> > >
> > > -    arp_nd_ns_match(ips, addr_family, &match);
> > > +    arp_nd_ns_match(ips, addr_family, &m);
> > > +    ds_clone(&match, &m);
> > > +
> > > +    bool has_cr_port = patch_op->cr_port;
> > > +
> > > +    /* If the patch_op has a chassis resident port, it means
> > > +     *    - its peer is a distributed gateway port (DGP) and
> > > +     *    - routing is centralized for the DGP's networks on
> > > +     *      the configured gateway chassis.
> > > +     *
> > > +     * If that's the case, make sure that the packets destined to
> > > +     * the DGP's MAC are sent to the chassis where the DGP resides.
> > > +     * */
> > > +
> > > +    if (has_cr_port) {
> > > +        ds_put_format(&match, " && is_chassis_resident(%s)",
> > > +                      patch_op->cr_port->json_key);
> > > +    }
> > >
> > >      /* Send a the packet to the router pipeline.  If the switch has non-router
> > >       * ports then flood it there as well.
> > > @@ -8179,6 +8276,31 @@ build_lswitch_rport_arp_req_flow(
> > >                                  lflow_ref);
> > >      }
> > >
> > > +    if (has_cr_port) {
> > > +        ds_clear(&match);
> > > +        ds_put_format(&match, "%s && !is_chassis_resident(%s)", ds_cstr(&m),
> > > +                      patch_op->cr_port->json_key);
> > > +        ds_clear(&actions);
> > > +        if (od->n_router_ports != od->nbs->n_ports) {
> > > +            ds_put_format(&actions, "clone {outport = %s; output; }; "
> > > +                                    "outport = \""MC_FLOOD_L2"\"; output;",
> > > +                          patch_op->cr_port->json_key);
> > > +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> > > +                                    priority, ds_cstr(&match),
> > > +                                    ds_cstr(&actions), stage_hint,
> > > +                                    lflow_ref);
> > > +        } else {
> > > +            ds_put_format(&actions, "outport = %s; output;",
> > > +                          patch_op->cr_port->json_key);
> > > +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
> > > +                                    priority, ds_cstr(&match),
> > > +                                    ds_cstr(&actions),
> > > +                                    stage_hint,
> > > +                                    lflow_ref);
> > > +        }
> > > +    }
> > > +
> > > +    ds_destroy(&m);
> > >      ds_destroy(&match);
> > >      ds_destroy(&actions);
> > >  }
> > > @@ -9548,7 +9670,11 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
> > >                                  struct ds *actions, struct ds *match)
> > >  {
> > >      ovs_assert(op->nbsp);
> > > -    if (lsp_is_external(op->nbsp)) {
> > > +
> > > +    /* Note: A switch port can also have a chassis resident derived port.
> > > +     * Check if 'op' is a chassis resident dervied port. If so, skip
> > > +     * adding unicast lookup flows for this port. */
> > > +    if (lsp_is_external(op->nbsp) || is_cr_port(op)) {
> > >          return;
> > >      }
> > >
> > > @@ -9566,8 +9692,6 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
> > >                             "outport = \""MC_UNKNOWN "\"; output;"
> > >                           : "outport = %s; output;")
> > >                           : debug_drop_action();
> > > -    ds_clear(actions);
> > > -    ds_put_format(actions, action, op->json_key);
> > >
> > >      if (lsp_is_router(op->nbsp) && op->peer && op->peer->nbrp) {
> > >          /* For ports connected to logical routers add flows to bypass the
> > > @@ -9614,14 +9738,43 @@ build_lswitch_ip_unicast_lookup(struct ovn_port *op,
> > >              if (add_chassis_resident_check) {
> > >                  ds_put_format(match, " && is_chassis_resident(%s)", json_key);
> > >              }
> > > +        } else if (op->cr_port) {
> > > +            /* If the op has a chassis resident port, it means
> > > +             *   - its peer is a distributed gateway port (DGP) and
> > > +             *   - routing is centralized for the DGP's networks on
> > > +             *     the configured gateway chassis.
> > > +             *
> > > +             * If that's the case, make sure that the packets destined to
> > > +             * the DGP's MAC are sent to the chassis where the DGP resides.
> > > +             * */
> > > +            ds_clear(actions);
> > > +            ds_put_format(actions, action, op->cr_port->json_key);
> > > +
> > > +            struct ds m = DS_EMPTY_INITIALIZER;
> > > +            ds_put_format(&m, "eth.dst == %s && !is_chassis_resident(%s)",
> > > +                          op->peer->lrp_networks.ea_s,
> > > +                          op->cr_port->json_key);
> > > +
> > > +            ovn_lflow_add_with_hint(lflows, op->od,
> > > +                                    S_SWITCH_IN_L2_LKUP, 50,
> > > +                                    ds_cstr(&m), ds_cstr(actions),
> > > +                                    &op->nbsp->header_,
> > > +                                    op->lflow_ref);
> > > +            ds_destroy(&m);
> > > +            ds_put_format(match, " && is_chassis_resident(%s)",
> > > +                          op->cr_port->json_key);
> > >          }
> > >
> > > +        ds_clear(actions);
> > > +        ds_put_format(actions, action, op->json_key);
> > >          ovn_lflow_add_with_hint(lflows, op->od,
> > >                                  S_SWITCH_IN_L2_LKUP, 50,
> > >                                  ds_cstr(match), ds_cstr(actions),
> > >                                  &op->nbsp->header_,
> > >                                  op->lflow_ref);
> > >      } else {
> > > +        ds_clear(actions);
> > > +        ds_put_format(actions, action, op->json_key);
> > >          for (size_t i = 0; i < op->n_lsp_addrs; i++) {
> > >              ds_clear(match);
> > >              ds_put_format(match, "eth.dst == %s", op->lsp_addrs[i].ea_s);
> > > @@ -11725,6 +11878,14 @@ build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op,
> > >          return;
> > >      }
> > >
> > > +    if (op->peer && op->peer->cr_port) {
> > > +        /* We don't add the below flows if the router port's peer has
> > > +         * a chassisresident port.  That's because routing is centralized on
> > > +         * the gateway chassis for the router port networks/subnets.
> > > +         */
> > > +        return;
> > > +    }
> > > +
> > >      /* Mac address to use when replying to ARP/NS. */
> > >      const char *mac_s = REG_INPORT_ETH_ADDR;
> > >      struct eth_addr mac;
> > > @@ -15109,6 +15270,16 @@ lrouter_check_nat_entry(const struct ovn_datapath *od,
> > >      /* For distributed router NAT, determine whether this NAT rule
> > >       * satisfies the conditions for distributed NAT processing. */
> > >      *distributed = false;
> > > +
> > > +    /* NAT cannnot be distributed if the DGP's peer
> > > +     * has a chassisresident port (as the routing is centralized
> > > +     * on the gateway chassis for the DGP's networks/subnets.)
> > > +     */
> > > +    struct ovn_port *l3dgw_port = *nat_l3dgw_port;
> > > +    if (l3dgw_port && l3dgw_port->peer && l3dgw_port->peer->cr_port) {
> > > +        return 0;
> > > +    }
> > > +
> > >      if (od->n_l3dgw_ports && !strcmp(nat->type, "dnat_and_snat") &&
> > >          nat->logical_port && nat->external_mac) {
> > >          if (eth_addr_from_string(nat->external_mac, mac)) {
> > > diff --git a/northd/northd.h b/northd/northd.h
> > > index d4a8d75abc..d7c9655916 100644
> > > --- a/northd/northd.h
> > > +++ b/northd/northd.h
> > > @@ -325,6 +325,7 @@ struct ovn_datapath {
> > >       * will be NULL. */
> > >      struct ovn_port **l3dgw_ports;
> > >      size_t n_l3dgw_ports;
> > > +    size_t n_allocated_l3dgw_ports;
> > >
> > >      /* router datapath has a logical port with redirect-type set to bridged. */
> > >      bool redirect_bridged;
> > > diff --git a/ovn-nb.xml b/ovn-nb.xml
> > > index a4362a4ef1..217d1cd1fe 100644
> > > --- a/ovn-nb.xml
> > > +++ b/ovn-nb.xml
> > > @@ -3499,6 +3499,40 @@ or
> > >            <ref column="options" key="gateway_mtu"/> option.
> > >          </p>
> > >        </column>
> > > +
> > > +      <column name="options" key="centralize_routing"
> > > +              type='{"type": "boolean"}'>
> > > +        <p>
> > > +          This option is applicable only if the router port is a
> > > +          distributed gateway port i.e if the <ref table="Logical_Router_Port"
> > > +          column="ha_chassis_group"/> column or
> > > +          <ref table="Logical_Router_Port" column="gateway_chassis"/>
> > > +          is set.
> > > +        </p>
> > > +
> > > +        <p>
> > > +          If set to <code>true</code>, routing for the router port's
> > > +          networks (set in the column <ref table="Logical_Router_Port"
> > > +          column="networks"/>) is centralized on the gateway chassis
> > > +          which claims this distributed gateway port.
> > > +        </p>
> > > +
> > > +        <p>
> > > +          Additionally for this option to take effect, below conditions
> > > +          must be met:
> > > +        </p>
> > > +
> > > +        <ul>
> > > +          <li>
> > > +            The Logical router has only one distributed gateway port.
> > > +          </li>
> > > +
> > > +          <li>
> > > +            The router port's peer logical switch has no localnet ports.
> > > +          </li>
> > > +
> > > +        </ul>
> > > +      </column>
> > >      </group>
> > >
> > >      <group title="Attachment">
> > > diff --git a/tests/multinode-macros.at b/tests/multinode-macros.at
> > > index 786e564860..757917626c 100644
> > > --- a/tests/multinode-macros.at
> > > +++ b/tests/multinode-macros.at
> > > @@ -92,7 +92,7 @@ m_count_rows() {
> > >  m_check_row_count() {
> > >      local db=$(parse_db $1) table=$(parse_table $1); shift
> > >      local count=$1; shift
> > > -    local found=$(m_count_rows $c $db:$table "$@")
> > > +    local found=$(m_count_rows $db:$table "$@")
> > >      echo
> > >      echo "Checking for $count rows in $db $table${1+ with $*}... found $found"
> > >      if test "$count" != "$found"; then
> > > diff --git a/tests/multinode.at b/tests/multinode.at
> > > index a7231130ac..a725414165 100644
> > > --- a/tests/multinode.at
> > > +++ b/tests/multinode.at
> > > @@ -1033,6 +1033,183 @@ M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [sh -c 'dd bs=512 count=2 if=/dev/uran
> > >  done
> > >  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ip route get 10.0.0.1 dev sw0p1 | grep -q 'mtu 942'])
> > >
> > > +# Reset back to geneve tunnels
> > > +for c in ovn-chassis-1 ovn-chassis-2 ovn-gw-1
> > > +do
> > > +    m_as $c ovs-vsctl set open . external-ids:ovn-encap-type=geneve
> > > +done
> > > +
> > > +AT_CLEANUP
> > > +
> > > +AT_SETUP([ovn multinode NAT on a provider network with no localnet ports])
> > > +
> > > +# Check that ovn-fake-multinode setup is up and running
> > > +check_fake_multinode_setup
> > > +
> > > +# Delete the multinode NB and OVS resources before starting the test.
> > > +cleanup_multinode_resources
> > > +
> > > +check multinode_nbctl ls-add sw0
> > > +check multinode_nbctl lsp-add sw0 sw0-port1
> > > +check multinode_nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03 10.0.0.3 1000::3"
> > > +check multinode_nbctl lsp-add sw0 sw0-port2
> > > +check multinode_nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:04 10.0.0.4 1000::4"
> > > +
> > > +m_as ovn-chassis-1 /data/create_fake_vm.sh sw0-port1 sw0p1 50:54:00:00:00:03 10.0.0.3 24 10.0.0.1 1000::3/64 1000::a
> > > +m_as ovn-chassis-2 /data/create_fake_vm.sh sw0-port2 sw0p2 50:54:00:00:00:04 10.0.0.4 24 10.0.0.1 1000::4/64 1000::a
> > > +
> > > +m_wait_for_ports_up
> > > +
> > > +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.4 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +# Create the second logical switch with one port
> > > +check multinode_nbctl ls-add sw1
> > > +check multinode_nbctl lsp-add sw1 sw1-port1
> > > +check multinode_nbctl lsp-set-addresses sw1-port1 "40:54:00:00:00:03 20.0.0.3 2000::3"
> > > +
> > > +# Create a logical router and attach both logical switches
> > > +check multinode_nbctl lr-add lr0
> > > +check multinode_nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 1000::a/64
> > > +check multinode_nbctl lsp-add sw0 sw0-lr0
> > > +check multinode_nbctl lsp-set-type sw0-lr0 router
> > > +check multinode_nbctl lsp-set-addresses sw0-lr0 router
> > > +check multinode_nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> > > +
> > > +check multinode_nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:02 20.0.0.1/24 2000::a/64
> > > +check multinode_nbctl lsp-add sw1 sw1-lr0
> > > +check multinode_nbctl lsp-set-type sw1-lr0 router
> > > +check multinode_nbctl lsp-set-addresses sw1-lr0 router
> > > +check multinode_nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> > > +
> > > +m_as ovn-chassis-2 /data/create_fake_vm.sh sw1-port1 sw1p1 40:54:00:00:00:03 20.0.0.3 24 20.0.0.1 2000::3/64 2000::a
> > > +
> > > +# create exteranl connection for N/S traffic
> > > +check multinode_nbctl ls-add public
> > > +check multinode_nbctl lsp-add public ln-public
> > > +check multinode_nbctl lsp-set-type ln-public localnet
> > > +check multinode_nbctl lsp-set-addresses ln-public unknown
> > > +check multinode_nbctl lsp-set-options ln-public network_name=public
> > > +
> > > +check multinode_nbctl lrp-add lr0 lr0-public 00:11:22:00:ff:01 172.20.0.100/24
> > > +check multinode_nbctl lsp-add public public-lr0
> > > +check multinode_nbctl lsp-set-type public-lr0 router
> > > +check multinode_nbctl lsp-set-addresses public-lr0 router
> > > +check multinode_nbctl lsp-set-options public-lr0 router-port=lr0-public
> > > +check multinode_nbctl lrp-set-gateway-chassis lr0-public ovn-gw-1 10
> > > +
> > > +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
> > > +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.120 20.0.0.3
> > > +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 10.0.0.0/24
> > > +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 20.0.0.0/24
> > > +
> > > +# Create a logical port pub-p1 and bind it in ovn-chassis-1
> > > +check multinode_nbctl lsp-add public public-port1
> > > +check multinode_nbctl lsp-set-addresses public-port1 "60:54:00:00:00:03 172.168.0.50"
> > > +
> > > +m_as ovn-chassis-1 /data/create_fake_vm.sh public-port1 pubp1 60:54:00:00:00:03 172.20.0.50 24 172.20.0.100
> > > +
> > > +check multinode_nbctl --wait=hv sync
> > > +
> > > +# First do basic ping tests before deleting the localnet port - ln-public.
> > > +# Once the localnet port is deleted from public ls, routing for 172.20.0.0/24
> > > +# is centralized on ovn-gw-1.
> > > +
> > > +# This function checks the North-South traffic.
> > > +run_ns_traffic() {
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.110], [ignore], [ignore])
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.120], [ignore], [ignore])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-2], [sw1p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  # Now ping from pubp1 to 172.20.0.100, 172.20.0.110, 172.20.0.120, 10.0.0.3 and 20.0.0.3
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.3 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 20.0.0.3 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +}
> > > +
> > > +# Test out the N-S traffic.
> > > +run_ns_traffic
> > > +
> > > +# Delete the localnet port by changing the type of ln-public to VIF port.
> > > +check multinode_nbctl --wait=hv lsp-set-type ln-public ""
> > > +
> > > +# cr-port should not be created for public-lr0 since the option
> > > +# centralize_routing=true is not yet set for lr0-public.
> > > +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > > +
> > > +# Set the option - centralize_routing now.
> > > +check multinode_nbctl --wait=hv set logical_router_port lr0-public options:centralize_routing=true
> > > +
> > > +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > > +m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> > > +
> > > +# Test out the N-S traffic.
> > > +run_ns_traffic
> > > +
> > > +# Re-add the localnet port
> > > +check multinode_nbctl --wait=hv lsp-set-type ln-public localnet
> > > +
> > > +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > > +
> > > +# Test out the N-S traffic.
> > > +run_ns_traffic
> > > +
> > > +# Delete the ln-public port this time.
> > > +check multinode_nbctl --wait=hv lsp-del ln-public
> > > +
> > > +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > > +m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> > > +
> > > +# Test out the N-S traffic.
> > > +run_ns_traffic
> > > +
> > >  AT_CLEANUP
> > >
> > >  AT_SETUP([ovn provider network - always_tunnel])
> > > diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
> > > index 57f89a7746..649c20f285 100644
> > > --- a/tests/ovn-northd.at
> > > +++ b/tests/ovn-northd.at
> > > @@ -2187,7 +2187,7 @@ match=(inport == "lrp-public" && arp.op == 1 && arp.tpa == 43.43.43.4 && is_chas
> > >  action=(eth.dst = eth.src; eth.src = 00:00:00:00:00:02; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 00:00:00:00:00:02; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > >  ])
> > >
> > > -# xreg0[0..47] isn't used anywhere else.
> > > +# xreg0[[0..47]] isn't used anywhere else.
> > >  AT_CHECK([ovn-sbctl lflow-list | grep "xreg0\[[0..47\]]" | grep -vE 'lr_in_admission|lr_in_ip_input'], [1], [])
> > >
> > >  AT_CLEANUP
> > > @@ -5503,13 +5503,14 @@ AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | grep "192.168.4.100" | grep "_MC_flo
> > >
> > >  AS_BOX([Configuring ro1-ls1 router port as a gateway router port])
> > >
> > > -ovn-nbctl --wait=sb lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> > > +check ovn-nbctl  lrp-set-gateway-chassis ro1-ls1 chassis-1 30
> > > +check ovn-nbctl --wait=sb lsp-add ls1 ln-ls1 -- lsp-set-type ln-ls1 localnet
> > >
> > >  ovn-sbctl lflow-list ls1 > ls1_lflows
> > >  AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | ovn_strip_lflows], [0], [dnl
> > >    table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> > >    table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> > > -  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01), action=(outport = "ls1-ro1"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01 && is_chassis_resident("cr-ro1-ls1")), action=(outport = "ls1-ro1"; output;)
> > >    table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:02), action=(outport = "vm1"; output;)
> > >    table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> > >    table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:01:01} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > > @@ -12475,3 +12476,512 @@ AT_CHECK([ovn-sbctl dump-flows lr | grep lr_in_dnat | ovn_strip_lflows], [0], [d
> > >
> > >  AT_CLEANUP
> > >  ])
> > > +
> > > +OVN_FOR_EACH_NORTHD_NO_HV([
> > > +AT_SETUP([NAT on a provider network with no localnet ports])
> > > +AT_KEYWORDS([NAT])
> > > +ovn_start
> > > +
> > > +check ovn-nbctl -- ls-add sw0 -- ls-add sw1
> > > +check ovn-nbctl lsp-add sw0 sw0-port1
> > > +check ovn-nbctl lr-add lr0
> > > +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
> > > +check ovn-nbctl lsp-add sw0 sw0-lr0
> > > +check ovn-nbctl lsp-set-type sw0-lr0 router
> > > +check ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01
> > > +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> > > +
> > > +check ovn-nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:03 20.0.0.1/24
> > > +check ovn-nbctl lsp-add sw1 sw1-lr0
> > > +check ovn-nbctl lsp-set-type sw1-lr0 router
> > > +check ovn-nbctl lsp-set-addresses sw1-lr0 router
> > > +check ovn-nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
> > > +
> > > +check ovn-sbctl chassis-add gw1 geneve 127.0.0.1
> > > +check ovn-nbctl ls-add public
> > > +check ovn-nbctl lsp-add public pub-p1
> > > +
> > > +# localnet port
> > > +check ovn-nbctl lsp-add public ln-public
> > > +check ovn-nbctl lsp-set-type ln-public localnet
> > > +check ovn-nbctl lsp-set-addresses ln-public unknown
> > > +check ovn-nbctl lsp-set-options ln-public network_name=public
> > > +
> > > +check ovn-nbctl lrp-add lr0 lr0-public 00:00:00:00:ff:02 172.168.0.10/24
> > > +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1
> > > +
> > > +check ovn-nbctl lsp-add public public-lr0
> > > +check ovn-nbctl lsp-set-type public-lr0 router
> > > +check ovn-nbctl lsp-set-addresses public-lr0 router
> > > +check ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public
> > > +
> > > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
> > > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.120 20.0.0.3
> > > +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 10.0.0.0/24
> > > +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 20.0.0.0/24
> > > +
> > > +check ovn-nbctl --wait=sb sync
> > > +
> > > +check_flows_no_cr_port_for_public_lr0() {
> > > +  # check that there is no port binding cr-public-lr0
> > > +  check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > > +
> > > +  ovn-sbctl dump-flows lr0 > lr0flows
> > > +  ovn-sbctl dump-flows public > publicflows
> > > +
> > > +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
> > > +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
> > > +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> > > +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > > +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> > > +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
> > > +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > > +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
> > > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> > > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +])
> > > +
> > > +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> > > +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-lr0-public")), action=(outport = "public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +])
> > > +
> > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > +])
> > > +
> > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +])
> > > +}
> > > +
> > > +check_flows_cr_port_for_public_lr0() {
> > > +  # check that there is port binding cr-public-lr0
> > > +  check_row_count Port_Binding 1 logical_port=cr-public-lr0
> > > +  check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
> > > +
> > > +  ovn-sbctl dump-flows lr0 > lr0flows
> > > +  ovn-sbctl dump-flows public > publicflows
> > > +
> > > +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
> > > +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
> > > +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
> > > +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > > +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
> > > +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
> > > +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
> > > +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
> > > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> > > +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > > +])
> > > +
> > > +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +])
> > > +
> > > +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
> > > +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-public-lr0")), action=(outport = "cr-public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-public-lr0")), action=(outport = "public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +])
> > > +
> > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > +])
> > > +
> > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +])
> > > +}
> > > +
> > > +# Check that the lflows are as expected when public has localnet port.
> > > +check_flows_no_cr_port_for_public_lr0
> > > +
> > > +# Remove the localnet port from public logical switch.
> > > +check ovn-nbctl --wait=sb lsp-set-type ln-public ""
> > > +
> > > +# Check that the lflows are as expected and there is no cr port
> > > +# created for "public-lr0"  when public has no localnet port
> > > +# since public doesn't have the option "overlay_provider_network=true"
> > > +# set.
> > > +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > > +
> > > +ovn-sbctl dump-flows lr0 > lr0flows
> > > +ovn-sbctl dump-flows public > publicflows
> > > +
> > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > +])
> > > +
> > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +])
> > > +
> > > +
> > > +# Set the option "centralize_routing=true" for lr0-public.
> > > +check ovn-nbctl --wait=sb set logical_router_port lr0-public options:centralize_routing=true
> > > +
> > > +# Check that the lflows are as expected and there is cr port created for public-lr0.
> > > +check_flows_cr_port_for_public_lr0
> > > +
> > > +# Set the type of ln-public back to localnet
> > > +check ovn-nbctl --wait=sb lsp-set-type ln-public localnet
> > > +
> > > +# Check that the lflows are as expected when public has localnet port.
> > > +check_flows_no_cr_port_for_public_lr0
> > > +
> > > +# Delete the localnet port
> > > +check ovn-nbctl --wait=sb lsp-del ln-public
> > > +
> > > +# Check that the lflows are as expected when public has no localnet port.
> > > +check_flows_cr_port_for_public_lr0
> > > +
> > > +# Create multiple gateway ports.  chassisresident port should not be
> > > +# created for 'public-lr0' even if there is no localnet port on 'public'
> > > +# logical switch.
> > > +check ovn-nbctl --wait=sb lrp-set-gateway-chassis lr0-sw0 gw1
> > > +# check that there is no port binding cr-public-lr0
> > > +check_row_count Port_Binding 0 logical_port=cr-public-lr0
> > > +
> > > +ovn-sbctl dump-flows lr0 > lr0flows
> > > +ovn-sbctl dump-flows public > publicflows
> > > +
> > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
> > > +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
> > > +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
> > > +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
> > > +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
> > > +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
> > > +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
> > > +])
> > > +
> > > +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
> > > +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
> > > +])
> > > +
> > > +AT_CLEANUP
> > > +])
> > > diff --git a/tests/ovn.at b/tests/ovn.at
> > > index 582ec56485..c468673f0c 100644
> > > --- a/tests/ovn.at
> > > +++ b/tests/ovn.at
> > > @@ -21227,10 +21227,10 @@ ovn-nbctl lsp-add sw0 rp-sw0 -- set Logical_Switch_Port rp-sw0 \
> > >      type=router options:router-port=sw0 \
> > >      -- lsp-set-addresses rp-sw0 router
> > >
> > > -ovn-nbctl lrp-add lr0 sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
> > > -    -- lrp-set-gateway-chassis sw1 hv2
> > > +ovn-nbctl lrp-add lr0 lr0-sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
> > > +    -- lrp-set-gateway-chassis lr0-sw1 hv2
> > >  ovn-nbctl lsp-add sw1 rp-sw1 -- set Logical_Switch_Port rp-sw1 \
> > > -    type=router options:router-port=sw1 \
> > > +    type=router options:router-port=lr0-sw1 \
> > >      -- lsp-set-addresses rp-sw1 router
> > >
> > >  ovn-nbctl lsp-add sw0 sw0-p0 \
> > > @@ -21242,6 +21242,8 @@ ovn-nbctl lsp-add sw0 sw0-p1 \
> > >  ovn-nbctl lsp-add sw1 sw1-p0 \
> > >      -- lsp-set-addresses sw1-p0 unknown
> > >
> > > +check ovn-nbctl lsp-add sw1 ln-sw1 -- lsp-set-type ln-sw1 localnet
> > > +
> > >  ovn-nbctl lr-nat-add lr0 snat 172.16.1.1 192.168.1.0/24
> > >  ovn-nbctl lr-nat-add lr0 snat 2002::1 2001::/64
> > >
> > > --
> > > 2.45.2
> > >
> > > _______________________________________________
> > > dev mailing list
> > > dev@openvswitch.org
> > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
> > _______________________________________________
> > dev mailing list
> > dev@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Felix Huettner Aug. 7, 2024, 2:18 p.m. UTC | #7 
On Wed, Aug 07, 2024 at 12:11:42AM -0700, Han Zhou wrote:
> [Einige Personen, die diese Nachricht erhalten haben, erhalten häufig keine E-Mails von zhouhan@gmail.com. Weitere Informationen, warum dies wichtig ist, finden Sie unter https://aka.ms/LearnAboutSenderIdentification ]
> 
> On Tue, Aug 6, 2024 at 8:35 PM Numan Siddique <numans@ovn.org> wrote:
> >
> > On Tue, Aug 6, 2024 at 2:50 AM Han Zhou <zhouhan@gmail.com> wrote:
> > >
> > > On Mon, Jul 29, 2024 at 7:39 PM <numans@ovn.org> wrote:
> > > >
> > > > From: Numan Siddique <numans@ovn.org>
> > > >
> > > > Consider a deployment with the below logical resources:
> > > >
> > > > 1. A bridged logical switch 'public' with a port - P1 and a localnet
> > > >    port ln-public.
> > > > 2. A logical router 'R'
> > > > 3. Logical switch 'public' connected to R via logical switch/router port
> > > >    peers (public-R and R-public).
> > > > 4. R-public is distributed gateway port with its network as 172.16.0.0/24
> > > > 5. NATs (dnat_and_snat) configured in 'R'.
> > > > 6. And a few overlay logical switches S1, S2 to R.
> > > >
> > > > Any traffic from logical port - P1 of public logical switch destined to
> > > > S1 or S2's logical ports goes out of the source chassis
> > > > (where P1 resides) via the localnet port and reaches the gateway chassis
> > > > which handles the routing.
> > > >
> > > > There are couple of traffic flow scenarios which doesn't work if the
> > > > logical switch 'public' doesn't have a localnet port.
> > > >
> > > > 1. Traffic from port - P1 destined to logical switches S1 or S2 gets
> > > >    dropped in the source chassis.  The packet enters the router R's
> > > >    pipeline, but it gets dropped in the 'lr_in_admission' stage since
> > > >    the logical flow to allow traffic destined to the distributed gateway
> > > >    port MAC is installed only on the gateway chassis.
> > > >
> > > > 2. NAT doesn't work as expected.
> > > >
> > > > In order to suppose this use case (of a logical switch not having a
> > > > localnet port, but has a distributed gateway port and NATs), this patch
> > > > supports the option 'centralize_routing', which can be configured on
> > > > the distributed gateway port (R-public in the example above).
> > > > If this option is set, then routing is centralized on the gateway
> > > > chassis for the traffic destined to the R-public's networks
> > > > (172.16.0.0/24 for the above example).  Traffic from P1 will be
> > > > tunnelled to the gateway chassis.
> > > >
> > > > ovn-northd creates a chassisresident port (cr-public-R) for the
> > > > logical switch port - public-R, along with cr-R-public inorder to
> > > > centralize the traffic.
> > > >
> > > Hi Numan, I have two questions here:
> > >
> > > > This feature gets enabled for the distributed gateway port R-public if
> > > >   - The above option is set to true in the R-public's options column.
> > >
> > > Why do we need an option to specify this behavior? If the LRP is a
> > > distributed gateway port and there is no localnet port on the
> > > connected LS, can't we just do the centralized routing?
> >
> > That was my initial approach.  But many test cases fail [1] with this approach.
> > It was hard to fix all these test cases and I was not sure if I'd
> > break any existing
> > scenarios.
> >
> > [1] https://github.com/numansiddique/ovn/actions/runs/10275831079/job/28435316245
> > testsuite: 75 76 161 162 163 164 165 166 167 168 169 170 183 184 185
> > 186 187 188 271 272 291 292 failed
> >
> 
> I am a little concerned about this. The original behavior is like a
> bug, because a LSP on the external overlay network is supposed to be
> able to communicate with the LSPs in the internal overlay networks. So
> I understand that this patch is trying to fix that. However, the fix
> would break some scenarios and it is not very clear why they break
> what scenarios would break. Because of this you added a new option to
> workaround the problem, because the test cases that didn't enable this
> option would still pass for sure. However, it also means that when
> this option is enabled, we are not very sure what other scenarios
> would break. Would it be better to examine the failed cases to
> understand what is really broken?

From my usage and tests all of these should be fixed by 
https://mail.openvswitch.org/pipermail/ovs-dev/2024-August/416264.html

Then i guess the option can be removed as well.

Thanks
Felix
Han Zhou Aug. 7, 2024, 3:35 p.m. UTC | #8 
On Wed, Aug 7, 2024 at 7:18 AM Felix Huettner
<felix.huettner@mail.schwarz> wrote:
>
> On Wed, Aug 07, 2024 at 12:11:42AM -0700, Han Zhou wrote:
> > [Einige Personen, die diese Nachricht erhalten haben, erhalten häufig keine E-Mails von zhouhan@gmail.com. Weitere Informationen, warum dies wichtig ist, finden Sie unter https://aka.ms/LearnAboutSenderIdentification ]
> >
> > On Tue, Aug 6, 2024 at 8:35 PM Numan Siddique <numans@ovn.org> wrote:
> > >
> > > On Tue, Aug 6, 2024 at 2:50 AM Han Zhou <zhouhan@gmail.com> wrote:
> > > >
> > > > On Mon, Jul 29, 2024 at 7:39 PM <numans@ovn.org> wrote:
> > > > >
> > > > > From: Numan Siddique <numans@ovn.org>
> > > > >
> > > > > Consider a deployment with the below logical resources:
> > > > >
> > > > > 1. A bridged logical switch 'public' with a port - P1 and a localnet
> > > > >    port ln-public.
> > > > > 2. A logical router 'R'
> > > > > 3. Logical switch 'public' connected to R via logical switch/router port
> > > > >    peers (public-R and R-public).
> > > > > 4. R-public is distributed gateway port with its network as 172.16.0.0/24
> > > > > 5. NATs (dnat_and_snat) configured in 'R'.
> > > > > 6. And a few overlay logical switches S1, S2 to R.
> > > > >
> > > > > Any traffic from logical port - P1 of public logical switch destined to
> > > > > S1 or S2's logical ports goes out of the source chassis
> > > > > (where P1 resides) via the localnet port and reaches the gateway chassis
> > > > > which handles the routing.
> > > > >
> > > > > There are couple of traffic flow scenarios which doesn't work if the
> > > > > logical switch 'public' doesn't have a localnet port.
> > > > >
> > > > > 1. Traffic from port - P1 destined to logical switches S1 or S2 gets
> > > > >    dropped in the source chassis.  The packet enters the router R's
> > > > >    pipeline, but it gets dropped in the 'lr_in_admission' stage since
> > > > >    the logical flow to allow traffic destined to the distributed gateway
> > > > >    port MAC is installed only on the gateway chassis.
> > > > >
> > > > > 2. NAT doesn't work as expected.
> > > > >
> > > > > In order to suppose this use case (of a logical switch not having a
> > > > > localnet port, but has a distributed gateway port and NATs), this patch
> > > > > supports the option 'centralize_routing', which can be configured on
> > > > > the distributed gateway port (R-public in the example above).
> > > > > If this option is set, then routing is centralized on the gateway
> > > > > chassis for the traffic destined to the R-public's networks
> > > > > (172.16.0.0/24 for the above example).  Traffic from P1 will be
> > > > > tunnelled to the gateway chassis.
> > > > >
> > > > > ovn-northd creates a chassisresident port (cr-public-R) for the
> > > > > logical switch port - public-R, along with cr-R-public inorder to
> > > > > centralize the traffic.
> > > > >
> > > > Hi Numan, I have two questions here:
> > > >
> > > > > This feature gets enabled for the distributed gateway port R-public if
> > > > >   - The above option is set to true in the R-public's options column.
> > > >
> > > > Why do we need an option to specify this behavior? If the LRP is a
> > > > distributed gateway port and there is no localnet port on the
> > > > connected LS, can't we just do the centralized routing?
> > >
> > > That was my initial approach.  But many test cases fail [1] with this approach.
> > > It was hard to fix all these test cases and I was not sure if I'd
> > > break any existing
> > > scenarios.
> > >
> > > [1] https://github.com/numansiddique/ovn/actions/runs/10275831079/job/28435316245
> > > testsuite: 75 76 161 162 163 164 165 166 167 168 169 170 183 184 185
> > > 186 187 188 271 272 291 292 failed
> > >
> >
> > I am a little concerned about this. The original behavior is like a
> > bug, because a LSP on the external overlay network is supposed to be
> > able to communicate with the LSPs in the internal overlay networks. So
> > I understand that this patch is trying to fix that. However, the fix
> > would break some scenarios and it is not very clear why they break
> > what scenarios would break. Because of this you added a new option to
> > workaround the problem, because the test cases that didn't enable this
> > option would still pass for sure. However, it also means that when
> > this option is enabled, we are not very sure what other scenarios
> > would break. Would it be better to examine the failed cases to
> > understand what is really broken?
>
> From my usage and tests all of these should be fixed by
> https://mail.openvswitch.org/pipermail/ovs-dev/2024-August/416264.html
>
> Then i guess the option can be removed as well.
>

Thanks Felix. Hope this solves the problem.

Numan, I don't have any other comment to the code except that the word
"chassisresident" in code (and also in the commit message) may be
confusing. We have been using the term chassisredirect or
chassis-redirect which is what "CR" stands for. I see only one
occurrence of chassisresident which is in the comment of is_cr_port()
and I believe that was a typo. So would be better to unify the term
and stick to chassisredirect in the var namings and comments.

I will wait for v3 and like to do some tests myself.

Thanks,
Han

> Thanks
> Felix
>
Mark Michelson Aug. 7, 2024, 5:24 p.m. UTC | #9 
I merged this to main.

On 8/2/24 14:10, Mark Michelson wrote:
> I know my Ack is already on this, but I had another look, and just to 
> reiterate:
> 
> Acked-by: Mark Michelson <mmichels@redhat.com>
> 
> On 7/29/24 22:38, numans@ovn.org wrote:
>> From: Numan Siddique <numans@ovn.org>
>>
>> Consider a deployment with the below logical resources:
>>
>> 1. A bridged logical switch 'public' with a port - P1 and a localnet
>>     port ln-public.
>> 2. A logical router 'R'
>> 3. Logical switch 'public' connected to R via logical switch/router port
>>     peers (public-R and R-public).
>> 4. R-public is distributed gateway port with its network as 172.16.0.0/24
>> 5. NATs (dnat_and_snat) configured in 'R'.
>> 6. And a few overlay logical switches S1, S2 to R.
>>
>> Any traffic from logical port - P1 of public logical switch destined to
>> S1 or S2's logical ports goes out of the source chassis
>> (where P1 resides) via the localnet port and reaches the gateway chassis
>> which handles the routing.
>>
>> There are couple of traffic flow scenarios which doesn't work if the
>> logical switch 'public' doesn't have a localnet port.
>>
>> 1. Traffic from port - P1 destined to logical switches S1 or S2 gets
>>     dropped in the source chassis.  The packet enters the router R's
>>     pipeline, but it gets dropped in the 'lr_in_admission' stage since
>>     the logical flow to allow traffic destined to the distributed gateway
>>     port MAC is installed only on the gateway chassis.
>>
>> 2. NAT doesn't work as expected.
>>
>> In order to suppose this use case (of a logical switch not having a
>> localnet port, but has a distributed gateway port and NATs), this patch
>> supports the option 'centralize_routing', which can be configured on
>> the distributed gateway port (R-public in the example above).
>> If this option is set, then routing is centralized on the gateway
>> chassis for the traffic destined to the R-public's networks
>> (172.16.0.0/24 for the above example).  Traffic from P1 will be
>> tunnelled to the gateway chassis.
>>
>> ovn-northd creates a chassisresident port (cr-public-R) for the
>> logical switch port - public-R, along with cr-R-public inorder to
>> centralize the traffic.
>>
>> This feature gets enabled for the distributed gateway port R-public if
>>    - The above option is set to true in the R-public's options column.
>>    - The logical switch 'public' doesn't have any localnet ports.
>>    - And R-public is the only distributed gateway port of R.
>>
>> Distributed NAT (i.e if external_mac and router_port is set) is
>> not supported and instead the router port mac is used for such traffic
>> and centralized on the gateway chassis.
>>
>> Reported-at: https://issues.redhat.com/browse/FDP-364
>> Acked-by: Mark Michelson <mmichels@redhat.com>
>> Signed-off-by: Numan Siddique <numans@ovn.org>
>> ---
>>
>> v1 -> v2
>> -------
>>     * Corrected the NEWS item entry for the new option.
>>     * Rebased and resolved conflicts.
>>
>> Note: This patch is the continuation from this series - 
>> https://patchwork.ozlabs.org/project/ovn/patch/20240606214432.168750-1-numans@ovn.org/
>>        Resetted the version number since the new option changed from LSP
>>        to LRP.
>>
>>
>>   NEWS                      |   3 +
>>   controller/physical.c     |   4 +
>>   northd/northd.c           | 257 +++++++++++++++----
>>   northd/northd.h           |   1 +
>>   ovn-nb.xml                |  34 +++
>>   tests/multinode-macros.at |   2 +-
>>   tests/multinode.at        | 177 +++++++++++++
>>   tests/ovn-northd.at       | 516 +++++++++++++++++++++++++++++++++++++-
>>   tests/ovn.at              |   8 +-
>>   9 files changed, 952 insertions(+), 50 deletions(-)
>>
>> diff --git a/NEWS b/NEWS
>> index 87e326f21e..8440a74677 100644
>> --- a/NEWS
>> +++ b/NEWS
>> @@ -47,6 +47,9 @@ Post v24.03.0
>>     - Add support for CT zone limit that can be specified per LR
>>       (options:ct-zone-limit), LS (other_config:ct-zone-limit) or LSP
>>       (options:ct-zone-limit).
>> +  - A new LRP option 'centralize_routing' has been added to a
>> +    distributed gateway port to centralize routing if the logical
>> +    switch of its peer doesn't have a localnet port.
>>   OVN v24.03.0 - 01 Mar 2024
>>   --------------------------
>> diff --git a/controller/physical.c b/controller/physical.c
>> index 3c0200c383..9e04ad5f22 100644
>> --- a/controller/physical.c
>> +++ b/controller/physical.c
>> @@ -1610,6 +1610,10 @@ consider_port_binding(struct ovsdb_idl_index 
>> *sbrec_port_binding_by_name,
>>                                                       ct_zones);
>>               put_zones_ofpacts(&zone_ids, ofpacts_p);
>> +            /* Clear the MFF_INPORT.  Its possible that the same 
>> packet may
>> +             * go out from the same tunnel inport. */
>> +            put_load(ofp_to_u16(OFPP_NONE), MFF_IN_PORT, 0, 16, 
>> ofpacts_p);
>> +
>>               /* Resubmit to table 41. */
>>               put_resubmit(OFTABLE_CHECK_LOOPBACK, ofpacts_p);
>>           }
>> diff --git a/northd/northd.c b/northd/northd.c
>> index 5c2fd74ff1..4f59d4f1a3 100644
>> --- a/northd/northd.c
>> +++ b/northd/northd.c
>> @@ -2107,6 +2107,55 @@ parse_lsp_addrs(struct ovn_port *op)
>>       }
>>   }
>> +static struct ovn_port *
>> +create_cr_port(struct ovn_port *op, struct hmap *ports,
>> +               struct ovs_list *both_dbs, struct ovs_list *nb_only)
>> +{
>> +    char *redirect_name = ovn_chassis_redirect_name(
>> +        op->nbsp ? op->nbsp->name : op->nbrp->name);
>> +
>> +    struct ovn_port *crp = ovn_port_find(ports, redirect_name);
>> +    if (crp && crp->sb && crp->sb->datapath == op->od->sb) {
>> +        ovn_port_set_nb(crp, NULL, op->nbrp);
>> +        ovs_list_remove(&crp->list);
>> +        ovs_list_push_back(both_dbs, &crp->list);
>> +    } else {
>> +        crp = ovn_port_create(ports, redirect_name,
>> +                              op->nbsp, op->nbrp, NULL);
>> +        ovs_list_push_back(nb_only, &crp->list);
>> +    }
>> +
>> +    crp->primary_port = op;
>> +    op->cr_port = crp;
>> +    crp->od = op->od;
>> +    free(redirect_name);
>> +
>> +    return crp;
>> +}
>> +
>> +/* Returns true if chassis resident port needs to be created for
>> + * op's peer logical switch.  False otherwise.
>> + *
>> + * Chassis resident port needs to be created if the following
>> + * conditionsd are met:
>> + *   - op is a distributed gateway port
>> + *   - op has the option 'centralize_routing' set to true
>> + *   - op is the only distributed gateway port attached to its
>> + *     router
>> + *   - op's peer logical switch has no localnet ports.
>> + */
>> +static bool
>> +peer_needs_cr_port_creation(struct ovn_port *op)
>> +{
>> +    if ((op->nbrp->n_gateway_chassis || op->nbrp->ha_chassis_group)
>> +        && op->od->n_l3dgw_ports == 1 && op->peer && op->peer->nbsp
>> +        && !op->peer->od->n_localnet_ports) {
>> +        return smap_get_bool(&op->nbrp->options, 
>> "centralize_routing", false);
>> +    }
>> +
>> +    return false;
>> +}
>> +
>>   static void
>>   join_logical_ports(const struct sbrec_port_binding_table 
>> *sbrec_pb_table,
>>                      struct hmap *ls_datapaths, struct hmap 
>> *lr_datapaths,
>> @@ -2214,9 +2263,10 @@ join_logical_ports(const struct 
>> sbrec_port_binding_table *sbrec_pb_table,
>>               tag_alloc_add_existing_tags(tag_alloc_table, nbsp);
>>           }
>>       }
>> +
>> +    struct hmapx dgps = HMAPX_INITIALIZER(&dgps);
>>       HMAP_FOR_EACH (od, key_node, lr_datapaths) {
>>           ovs_assert(od->nbr);
>> -        size_t n_allocated_l3dgw_ports = 0;
>>           for (size_t i = 0; i < od->nbr->n_ports; i++) {
>>               const struct nbrec_logical_router_port *nbrp
>>                   = od->nbr->ports[i];
>> @@ -2280,10 +2330,7 @@ join_logical_ports(const struct 
>> sbrec_port_binding_table *sbrec_pb_table,
>>                       redirect_type && !strcasecmp(redirect_type, 
>> "bridged");
>>               }
>> -            if (op->nbrp->ha_chassis_group ||
>> -                op->nbrp->n_gateway_chassis) {
>> -                /* Additional "derived" ovn_port crp represents the
>> -                 * instance of op on the gateway chassis. */
>> +            if (op->nbrp->ha_chassis_group || 
>> op->nbrp->n_gateway_chassis) {
>>                   const char *gw_chassis = 
>> smap_get(&op->od->nbr->options,
>>                                                  "chassis");
>>                   if (gw_chassis) {
>> @@ -2292,34 +2339,9 @@ join_logical_ports(const struct 
>> sbrec_port_binding_table *sbrec_pb_table,
>>                       VLOG_WARN_RL(&rl, "Bad configuration: distributed "
>>                                    "gateway port configured on port %s "
>>                                    "on L3 gateway router", nbrp->name);
>> -                    continue;
>> -                }
>> -
>> -                char *redirect_name =
>> -                    ovn_chassis_redirect_name(nbrp->name);
>> -                struct ovn_port *crp = ovn_port_find(ports, 
>> redirect_name);
>> -                if (crp && crp->sb && crp->sb->datapath == od->sb) {
>> -                    ovn_port_set_nb(crp, NULL, nbrp);
>> -                    ovs_list_remove(&crp->list);
>> -                    ovs_list_push_back(both, &crp->list);
>>                   } else {
>> -                    crp = ovn_port_create(ports, redirect_name,
>> -                                          NULL, nbrp, NULL);
>> -                    ovs_list_push_back(nb_only, &crp->list);
>> -                }
>> -                crp->primary_port = op;
>> -                op->cr_port = crp;
>> -                crp->od = od;
>> -                free(redirect_name);
>> -
>> -                /* Add to l3dgw_ports in od, for later use during flow
>> -                 * creation. */
>> -                if (od->n_l3dgw_ports == n_allocated_l3dgw_ports) {
>> -                    od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
>> -                                                 
>> &n_allocated_l3dgw_ports,
>> -                                                 sizeof 
>> *od->l3dgw_ports);
>> +                    hmapx_add(&dgps, op);
>>                   }
>> -                od->l3dgw_ports[od->n_l3dgw_ports++] = op;
>>              }
>>           }
>>       }
>> @@ -2376,12 +2398,6 @@ join_logical_ports(const struct 
>> sbrec_port_binding_table *sbrec_pb_table,
>>                           arp_proxy, op->nbsp->name);
>>                   }
>>               }
>> -
>> -            /* Only used for the router type LSP whose peer is 
>> l3dgw_port */
>> -            if (op->peer && is_l3dgw_port(op->peer)) {
>> -                op->enable_router_port_acl = smap_get_bool(
>> -                    &op->nbsp->options, "enable_router_port_acl", 
>> false);
>> -            }
>>           } else if (op->nbrp && op->nbrp->peer && !is_cr_port(op)) {
>>               struct ovn_port *peer = ovn_port_find(ports, 
>> op->nbrp->peer);
>>               if (peer) {
>> @@ -2402,6 +2418,57 @@ join_logical_ports(const struct 
>> sbrec_port_binding_table *sbrec_pb_table,
>>           }
>>       }
>> +    struct hmapx_node *hmapx_node;
>> +    HMAPX_FOR_EACH (hmapx_node, &dgps) {
>> +        op = hmapx_node->data;
>> +        od = op->od;
>> +        ovs_assert(op->nbrp);
>> +        ovs_assert(op->nbrp->ha_chassis_group || 
>> op->nbrp->n_gateway_chassis);
>> +
>> +        /* Additional "derived" ovn_port crp represents the instance 
>> of op on
>> +         * the gateway chassis. */
>> +        struct ovn_port *crp = create_cr_port(op, ports, both, nb_only);
>> +        ovs_assert(crp);
>> +
>> +        /* Add to l3dgw_ports in od, for later use during flow 
>> creation. */
>> +        if (od->n_l3dgw_ports == od->n_allocated_l3dgw_ports) {
>> +            od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
>> +                                        &od->n_allocated_l3dgw_ports,
>> +                                        sizeof *od->l3dgw_ports);
>> +        }
>> +        od->l3dgw_ports[od->n_l3dgw_ports++] = op;
>> +
>> +        if (op->peer && op->peer->nbsp) {
>> +            /* Only used for the router type LSP whose peer is 
>> l3dgw_port */
>> +            op->peer->enable_router_port_acl = smap_get_bool(
>> +                    &op->peer->nbsp->options, 
>> "enable_router_port_acl", false);
>> +        }
>> +    }
>> +
>> +
>> +    /* Create chassisresident port for the distributed gateway port's 
>> (DGP)
>> +     * peer if
>> +     *  - DGP's router has only one DGP and
>> +     *  - Its peer is a logical switch port and
>> +     *  - It's peer's logical switch has no localnet ports and
>> +     *  - option 'centralize_routing' is set to true for the DGP.
>> +     *
>> +     * This is required to support
>> +     *   - NAT via geneve (for the overlay provider networks) and
>> +     *   - to centralize routing on the gateway chassis for the traffic
>> +     *     destined to the DGP's networks.
>> +     *
>> +     * Future enhancement: Support 'centralizerouting' for all the DGP's
>> +     * of a logical router.
>> +     * */
>> +    HMAPX_FOR_EACH (hmapx_node, &dgps) {
>> +        op = hmapx_node->data;
>> +        if (peer_needs_cr_port_creation(op)) {
>> +            create_cr_port(op->peer, ports, both, nb_only);
>> +        }
>> +    }
>> +    hmapx_destroy(&dgps);
>> +
>>       /* Wait until all ports have been connected to add to IPAM since
>>        * it relies on proper peers to be set
>>        */
>> @@ -3184,16 +3251,28 @@ ovn_port_update_sbrec(struct ovsdb_idl_txn 
>> *ovnsb_txn,
>>                * type "l3gateway". */
>>               if (chassis) {
>>                   sbrec_port_binding_set_type(op->sb, "l3gateway");
>> +            } else if (is_cr_port(op)) {
>> +                sbrec_port_binding_set_type(op->sb, "chassisredirect");
>> +                ovs_assert(op->primary_port->peer);
>> +                ovs_assert(op->primary_port->peer->cr_port);
>> +                ovs_assert(op->primary_port->peer->cr_port->sb);
>> +                sbrec_port_binding_set_ha_chassis_group(
>> +                    op->sb,
>> +                    
>> op->primary_port->peer->cr_port->sb->ha_chassis_group);
>> +
>>               } else {
>>                   sbrec_port_binding_set_type(op->sb, "patch");
>>               }
>>               const char *router_port = smap_get(&op->nbsp->options,
>>                                                  "router-port");
>> -            if (router_port || chassis) {
>> +            if (router_port || chassis || is_cr_port(op)) {
>>                   struct smap new;
>>                   smap_init(&new);
>> -                if (router_port) {
>> +
>> +                if (is_cr_port(op)) {
>> +                    smap_add(&new, "distributed-port", op->nbsp->name);
>> +                } else if (router_port) {
>>                       smap_add(&new, "peer", router_port);
>>                   }
>>                   if (chassis) {
>> @@ -8155,9 +8234,27 @@ build_lswitch_rport_arp_req_flow(
>>       struct lflow_ref *lflow_ref)
>>   {
>>       struct ds match   = DS_EMPTY_INITIALIZER;
>> +    struct ds m       = DS_EMPTY_INITIALIZER;
>>       struct ds actions = DS_EMPTY_INITIALIZER;
>> -    arp_nd_ns_match(ips, addr_family, &match);
>> +    arp_nd_ns_match(ips, addr_family, &m);
>> +    ds_clone(&match, &m);
>> +
>> +    bool has_cr_port = patch_op->cr_port;
>> +
>> +    /* If the patch_op has a chassis resident port, it means
>> +     *    - its peer is a distributed gateway port (DGP) and
>> +     *    - routing is centralized for the DGP's networks on
>> +     *      the configured gateway chassis.
>> +     *
>> +     * If that's the case, make sure that the packets destined to
>> +     * the DGP's MAC are sent to the chassis where the DGP resides.
>> +     * */
>> +
>> +    if (has_cr_port) {
>> +        ds_put_format(&match, " && is_chassis_resident(%s)",
>> +                      patch_op->cr_port->json_key);
>> +    }
>>       /* Send a the packet to the router pipeline.  If the switch has 
>> non-router
>>        * ports then flood it there as well.
>> @@ -8179,6 +8276,31 @@ build_lswitch_rport_arp_req_flow(
>>                                   lflow_ref);
>>       }
>> +    if (has_cr_port) {
>> +        ds_clear(&match);
>> +        ds_put_format(&match, "%s && !is_chassis_resident(%s)", 
>> ds_cstr(&m),
>> +                      patch_op->cr_port->json_key);
>> +        ds_clear(&actions);
>> +        if (od->n_router_ports != od->nbs->n_ports) {
>> +            ds_put_format(&actions, "clone {outport = %s; output; }; "
>> +                                    "outport = \""MC_FLOOD_L2"\"; 
>> output;",
>> +                          patch_op->cr_port->json_key);
>> +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
>> +                                    priority, ds_cstr(&match),
>> +                                    ds_cstr(&actions), stage_hint,
>> +                                    lflow_ref);
>> +        } else {
>> +            ds_put_format(&actions, "outport = %s; output;",
>> +                          patch_op->cr_port->json_key);
>> +            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
>> +                                    priority, ds_cstr(&match),
>> +                                    ds_cstr(&actions),
>> +                                    stage_hint,
>> +                                    lflow_ref);
>> +        }
>> +    }
>> +
>> +    ds_destroy(&m);
>>       ds_destroy(&match);
>>       ds_destroy(&actions);
>>   }
>> @@ -9548,7 +9670,11 @@ build_lswitch_ip_unicast_lookup(struct ovn_port 
>> *op,
>>                                   struct ds *actions, struct ds *match)
>>   {
>>       ovs_assert(op->nbsp);
>> -    if (lsp_is_external(op->nbsp)) {
>> +
>> +    /* Note: A switch port can also have a chassis resident derived 
>> port.
>> +     * Check if 'op' is a chassis resident dervied port. If so, skip
>> +     * adding unicast lookup flows for this port. */
>> +    if (lsp_is_external(op->nbsp) || is_cr_port(op)) {
>>           return;
>>       }
>> @@ -9566,8 +9692,6 @@ build_lswitch_ip_unicast_lookup(struct ovn_port 
>> *op,
>>                              "outport = \""MC_UNKNOWN "\"; output;"
>>                            : "outport = %s; output;")
>>                            : debug_drop_action();
>> -    ds_clear(actions);
>> -    ds_put_format(actions, action, op->json_key);
>>       if (lsp_is_router(op->nbsp) && op->peer && op->peer->nbrp) {
>>           /* For ports connected to logical routers add flows to 
>> bypass the
>> @@ -9614,14 +9738,43 @@ build_lswitch_ip_unicast_lookup(struct 
>> ovn_port *op,
>>               if (add_chassis_resident_check) {
>>                   ds_put_format(match, " && is_chassis_resident(%s)", 
>> json_key);
>>               }
>> +        } else if (op->cr_port) {
>> +            /* If the op has a chassis resident port, it means
>> +             *   - its peer is a distributed gateway port (DGP) and
>> +             *   - routing is centralized for the DGP's networks on
>> +             *     the configured gateway chassis.
>> +             *
>> +             * If that's the case, make sure that the packets 
>> destined to
>> +             * the DGP's MAC are sent to the chassis where the DGP 
>> resides.
>> +             * */
>> +            ds_clear(actions);
>> +            ds_put_format(actions, action, op->cr_port->json_key);
>> +
>> +            struct ds m = DS_EMPTY_INITIALIZER;
>> +            ds_put_format(&m, "eth.dst == %s && 
>> !is_chassis_resident(%s)",
>> +                          op->peer->lrp_networks.ea_s,
>> +                          op->cr_port->json_key);
>> +
>> +            ovn_lflow_add_with_hint(lflows, op->od,
>> +                                    S_SWITCH_IN_L2_LKUP, 50,
>> +                                    ds_cstr(&m), ds_cstr(actions),
>> +                                    &op->nbsp->header_,
>> +                                    op->lflow_ref);
>> +            ds_destroy(&m);
>> +            ds_put_format(match, " && is_chassis_resident(%s)",
>> +                          op->cr_port->json_key);
>>           }
>> +        ds_clear(actions);
>> +        ds_put_format(actions, action, op->json_key);
>>           ovn_lflow_add_with_hint(lflows, op->od,
>>                                   S_SWITCH_IN_L2_LKUP, 50,
>>                                   ds_cstr(match), ds_cstr(actions),
>>                                   &op->nbsp->header_,
>>                                   op->lflow_ref);
>>       } else {
>> +        ds_clear(actions);
>> +        ds_put_format(actions, action, op->json_key);
>>           for (size_t i = 0; i < op->n_lsp_addrs; i++) {
>>               ds_clear(match);
>>               ds_put_format(match, "eth.dst == %s", 
>> op->lsp_addrs[i].ea_s);
>> @@ -11725,6 +11878,14 @@ build_lrouter_port_nat_arp_nd_flow(struct 
>> ovn_port *op,
>>           return;
>>       }
>> +    if (op->peer && op->peer->cr_port) {
>> +        /* We don't add the below flows if the router port's peer has
>> +         * a chassisresident port.  That's because routing is 
>> centralized on
>> +         * the gateway chassis for the router port networks/subnets.
>> +         */
>> +        return;
>> +    }
>> +
>>       /* Mac address to use when replying to ARP/NS. */
>>       const char *mac_s = REG_INPORT_ETH_ADDR;
>>       struct eth_addr mac;
>> @@ -15109,6 +15270,16 @@ lrouter_check_nat_entry(const struct 
>> ovn_datapath *od,
>>       /* For distributed router NAT, determine whether this NAT rule
>>        * satisfies the conditions for distributed NAT processing. */
>>       *distributed = false;
>> +
>> +    /* NAT cannnot be distributed if the DGP's peer
>> +     * has a chassisresident port (as the routing is centralized
>> +     * on the gateway chassis for the DGP's networks/subnets.)
>> +     */
>> +    struct ovn_port *l3dgw_port = *nat_l3dgw_port;
>> +    if (l3dgw_port && l3dgw_port->peer && l3dgw_port->peer->cr_port) {
>> +        return 0;
>> +    }
>> +
>>       if (od->n_l3dgw_ports && !strcmp(nat->type, "dnat_and_snat") &&
>>           nat->logical_port && nat->external_mac) {
>>           if (eth_addr_from_string(nat->external_mac, mac)) {
>> diff --git a/northd/northd.h b/northd/northd.h
>> index d4a8d75abc..d7c9655916 100644
>> --- a/northd/northd.h
>> +++ b/northd/northd.h
>> @@ -325,6 +325,7 @@ struct ovn_datapath {
>>        * will be NULL. */
>>       struct ovn_port **l3dgw_ports;
>>       size_t n_l3dgw_ports;
>> +    size_t n_allocated_l3dgw_ports;
>>       /* router datapath has a logical port with redirect-type set to 
>> bridged. */
>>       bool redirect_bridged;
>> diff --git a/ovn-nb.xml b/ovn-nb.xml
>> index a4362a4ef1..217d1cd1fe 100644
>> --- a/ovn-nb.xml
>> +++ b/ovn-nb.xml
>> @@ -3499,6 +3499,40 @@ or
>>             <ref column="options" key="gateway_mtu"/> option.
>>           </p>
>>         </column>
>> +
>> +      <column name="options" key="centralize_routing"
>> +              type='{"type": "boolean"}'>
>> +        <p>
>> +          This option is applicable only if the router port is a
>> +          distributed gateway port i.e if the <ref 
>> table="Logical_Router_Port"
>> +          column="ha_chassis_group"/> column or
>> +          <ref table="Logical_Router_Port" column="gateway_chassis"/>
>> +          is set.
>> +        </p>
>> +
>> +        <p>
>> +          If set to <code>true</code>, routing for the router port's
>> +          networks (set in the column <ref table="Logical_Router_Port"
>> +          column="networks"/>) is centralized on the gateway chassis
>> +          which claims this distributed gateway port.
>> +        </p>
>> +
>> +        <p>
>> +          Additionally for this option to take effect, below conditions
>> +          must be met:
>> +        </p>
>> +
>> +        <ul>
>> +          <li>
>> +            The Logical router has only one distributed gateway port.
>> +          </li>
>> +
>> +          <li>
>> +            The router port's peer logical switch has no localnet ports.
>> +          </li>
>> +
>> +        </ul>
>> +      </column>
>>       </group>
>>       <group title="Attachment">
>> diff --git a/tests/multinode-macros.at b/tests/multinode-macros.at
>> index 786e564860..757917626c 100644
>> --- a/tests/multinode-macros.at
>> +++ b/tests/multinode-macros.at
>> @@ -92,7 +92,7 @@ m_count_rows() {
>>   m_check_row_count() {
>>       local db=$(parse_db $1) table=$(parse_table $1); shift
>>       local count=$1; shift
>> -    local found=$(m_count_rows $c $db:$table "$@")
>> +    local found=$(m_count_rows $db:$table "$@")
>>       echo
>>       echo "Checking for $count rows in $db $table${1+ with $*}... 
>> found $found"
>>       if test "$count" != "$found"; then
>> diff --git a/tests/multinode.at b/tests/multinode.at
>> index a7231130ac..a725414165 100644
>> --- a/tests/multinode.at
>> +++ b/tests/multinode.at
>> @@ -1033,6 +1033,183 @@ M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [sh 
>> -c 'dd bs=512 count=2 if=/dev/uran
>>   done
>>   M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ip route get 10.0.0.1 dev 
>> sw0p1 | grep -q 'mtu 942'])
>> +# Reset back to geneve tunnels
>> +for c in ovn-chassis-1 ovn-chassis-2 ovn-gw-1
>> +do
>> +    m_as $c ovs-vsctl set open . external-ids:ovn-encap-type=geneve
>> +done
>> +
>> +AT_CLEANUP
>> +
>> +AT_SETUP([ovn multinode NAT on a provider network with no localnet 
>> ports])
>> +
>> +# Check that ovn-fake-multinode setup is up and running
>> +check_fake_multinode_setup
>> +
>> +# Delete the multinode NB and OVS resources before starting the test.
>> +cleanup_multinode_resources
>> +
>> +check multinode_nbctl ls-add sw0
>> +check multinode_nbctl lsp-add sw0 sw0-port1
>> +check multinode_nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03 
>> 10.0.0.3 1000::3"
>> +check multinode_nbctl lsp-add sw0 sw0-port2
>> +check multinode_nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:04 
>> 10.0.0.4 1000::4"
>> +
>> +m_as ovn-chassis-1 /data/create_fake_vm.sh sw0-port1 sw0p1 
>> 50:54:00:00:00:03 10.0.0.3 24 10.0.0.1 1000::3/64 1000::a
>> +m_as ovn-chassis-2 /data/create_fake_vm.sh sw0-port2 sw0p2 
>> 50:54:00:00:00:04 10.0.0.4 24 10.0.0.1 1000::4/64 1000::a
>> +
>> +m_wait_for_ports_up
>> +
>> +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 
>> 10.0.0.4 | FORMAT_PING], \
>> +[0], [dnl
>> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
>> +])
>> +
>> +# Create the second logical switch with one port
>> +check multinode_nbctl ls-add sw1
>> +check multinode_nbctl lsp-add sw1 sw1-port1
>> +check multinode_nbctl lsp-set-addresses sw1-port1 "40:54:00:00:00:03 
>> 20.0.0.3 2000::3"
>> +
>> +# Create a logical router and attach both logical switches
>> +check multinode_nbctl lr-add lr0
>> +check multinode_nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 
>> 10.0.0.1/24 1000::a/64
>> +check multinode_nbctl lsp-add sw0 sw0-lr0
>> +check multinode_nbctl lsp-set-type sw0-lr0 router
>> +check multinode_nbctl lsp-set-addresses sw0-lr0 router
>> +check multinode_nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
>> +
>> +check multinode_nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:02 
>> 20.0.0.1/24 2000::a/64
>> +check multinode_nbctl lsp-add sw1 sw1-lr0
>> +check multinode_nbctl lsp-set-type sw1-lr0 router
>> +check multinode_nbctl lsp-set-addresses sw1-lr0 router
>> +check multinode_nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
>> +
>> +m_as ovn-chassis-2 /data/create_fake_vm.sh sw1-port1 sw1p1 
>> 40:54:00:00:00:03 20.0.0.3 24 20.0.0.1 2000::3/64 2000::a
>> +
>> +# create exteranl connection for N/S traffic
>> +check multinode_nbctl ls-add public
>> +check multinode_nbctl lsp-add public ln-public
>> +check multinode_nbctl lsp-set-type ln-public localnet
>> +check multinode_nbctl lsp-set-addresses ln-public unknown
>> +check multinode_nbctl lsp-set-options ln-public network_name=public
>> +
>> +check multinode_nbctl lrp-add lr0 lr0-public 00:11:22:00:ff:01 
>> 172.20.0.100/24
>> +check multinode_nbctl lsp-add public public-lr0
>> +check multinode_nbctl lsp-set-type public-lr0 router
>> +check multinode_nbctl lsp-set-addresses public-lr0 router
>> +check multinode_nbctl lsp-set-options public-lr0 router-port=lr0-public
>> +check multinode_nbctl lrp-set-gateway-chassis lr0-public ovn-gw-1 10
>> +
>> +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.110 
>> 10.0.0.3 sw0-port1 30:54:00:00:00:03
>> +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.120 20.0.0.3
>> +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 10.0.0.0/24
>> +check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 20.0.0.0/24
>> +
>> +# Create a logical port pub-p1 and bind it in ovn-chassis-1
>> +check multinode_nbctl lsp-add public public-port1
>> +check multinode_nbctl lsp-set-addresses public-port1 
>> "60:54:00:00:00:03 172.168.0.50"
>> +
>> +m_as ovn-chassis-1 /data/create_fake_vm.sh public-port1 pubp1 
>> 60:54:00:00:00:03 172.20.0.50 24 172.20.0.100
>> +
>> +check multinode_nbctl --wait=hv sync
>> +
>> +# First do basic ping tests before deleting the localnet port - 
>> ln-public.
>> +# Once the localnet port is deleted from public ls, routing for 
>> 172.20.0.0/24
>> +# is centralized on ovn-gw-1.
>> +
>> +# This function checks the North-South traffic.
>> +run_ns_traffic() {
>> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.110], 
>> [ignore], [ignore])
>> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.120], 
>> [ignore], [ignore])
>> +
>> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 
>> 172.20.0.100 | FORMAT_PING], \
>> +[0], [dnl
>> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
>> +])
>> +
>> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 
>> 172.20.0.110 | FORMAT_PING], \
>> +[0], [dnl
>> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
>> +])
>> +
>> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 
>> 172.20.0.120 | FORMAT_PING], \
>> +[0], [dnl
>> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
>> +])
>> +
>> +  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 
>> 172.20.0.50 | FORMAT_PING], \
>> +[0], [dnl
>> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
>> +])
>> +
>> +  M_NS_CHECK_EXEC([ovn-chassis-2], [sw1p1], [ping -q -c 3 -i 0.3 -w 2 
>> 172.20.0.50 | FORMAT_PING], \
>> +[0], [dnl
>> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
>> +])
>> +
>> +  # Now ping from pubp1 to 172.20.0.100, 172.20.0.110, 172.20.0.120, 
>> 10.0.0.3 and 20.0.0.3
>> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 
>> 172.20.0.100 | FORMAT_PING], \
>> +[0], [dnl
>> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
>> +])
>> +
>> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 
>> 172.20.0.110 | FORMAT_PING], \
>> +[0], [dnl
>> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
>> +])
>> +
>> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 
>> 172.20.0.120 | FORMAT_PING], \
>> +[0], [dnl
>> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
>> +])
>> +
>> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 
>> 10.0.0.3 | FORMAT_PING], \
>> +[0], [dnl
>> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
>> +])
>> +
>> +  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 
>> 20.0.0.3 | FORMAT_PING], \
>> +[0], [dnl
>> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
>> +])
>> +}
>> +
>> +# Test out the N-S traffic.
>> +run_ns_traffic
>> +
>> +# Delete the localnet port by changing the type of ln-public to VIF 
>> port.
>> +check multinode_nbctl --wait=hv lsp-set-type ln-public ""
>> +
>> +# cr-port should not be created for public-lr0 since the option
>> +# centralize_routing=true is not yet set for lr0-public.
>> +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
>> +
>> +# Set the option - centralize_routing now.
>> +check multinode_nbctl --wait=hv set logical_router_port lr0-public 
>> options:centralize_routing=true
>> +
>> +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
>> +m_check_column chassisredirect Port_Binding type 
>> logical_port=cr-public-lr0
>> +
>> +# Test out the N-S traffic.
>> +run_ns_traffic
>> +
>> +# Re-add the localnet port
>> +check multinode_nbctl --wait=hv lsp-set-type ln-public localnet
>> +
>> +m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
>> +
>> +# Test out the N-S traffic.
>> +run_ns_traffic
>> +
>> +# Delete the ln-public port this time.
>> +check multinode_nbctl --wait=hv lsp-del ln-public
>> +
>> +m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
>> +m_check_column chassisredirect Port_Binding type 
>> logical_port=cr-public-lr0
>> +
>> +# Test out the N-S traffic.
>> +run_ns_traffic
>> +
>>   AT_CLEANUP
>>   AT_SETUP([ovn provider network - always_tunnel])
>> diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
>> index 57f89a7746..649c20f285 100644
>> --- a/tests/ovn-northd.at
>> +++ b/tests/ovn-northd.at
>> @@ -2187,7 +2187,7 @@ match=(inport == "lrp-public" && arp.op == 1 && 
>> arp.tpa == 43.43.43.4 && is_chas
>>   action=(eth.dst = eth.src; eth.src = 00:00:00:00:00:02; arp.op = 2; 
>> /* ARP reply */ arp.tha = arp.sha; arp.sha = 00:00:00:00:00:02; 
>> arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
>>   ])
>> -# xreg0[0..47] isn't used anywhere else.
>> +# xreg0[[0..47]] isn't used anywhere else.
>>   AT_CHECK([ovn-sbctl lflow-list | grep "xreg0\[[0..47\]]" | grep -vE 
>> 'lr_in_admission|lr_in_ip_input'], [1], [])
>>   AT_CLEANUP
>> @@ -5503,13 +5503,14 @@ AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | 
>> grep "192.168.4.100" | grep "_MC_flo
>>   AS_BOX([Configuring ro1-ls1 router port as a gateway router port])
>> -ovn-nbctl --wait=sb lrp-set-gateway-chassis ro1-ls1 chassis-1 30
>> +check ovn-nbctl  lrp-set-gateway-chassis ro1-ls1 chassis-1 30
>> +check ovn-nbctl --wait=sb lsp-add ls1 ln-ls1 -- lsp-set-type ln-ls1 
>> localnet
>>   ovn-sbctl lflow-list ls1 > ls1_lflows
>>   AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | ovn_strip_lflows], [0], 
>> [dnl
>>     table=??(ls_in_l2_lkup      ), priority=0    , match=(1), 
>> action=(outport = get_fdb(eth.dst); next;)
>>     table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == 
>> $svc_monitor_mac && (tcp || icmp || icmp6)), 
>> action=(handle_svc_check(inport);)
>> -  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 
>> 00:00:00:00:01:01), action=(outport = "ls1-ro1"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 
>> 00:00:00:00:01:01 && is_chassis_resident("cr-ro1-ls1")), 
>> action=(outport = "ls1-ro1"; output;)
>>     table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 
>> 00:00:00:00:01:02), action=(outport = "vm1"; output;)
>>     table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), 
>> action=(outport = "_MC_flood"; output;)
>>     table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == 
>> {00:00:00:00:01:01} && (arp.op == 1 || rarp.op == 3 || nd_ns)), 
>> action=(outport = "_MC_flood_l2"; output;)
>> @@ -12475,3 +12476,512 @@ AT_CHECK([ovn-sbctl dump-flows lr | grep 
>> lr_in_dnat | ovn_strip_lflows], [0], [d
>>   AT_CLEANUP
>>   ])
>> +
>> +OVN_FOR_EACH_NORTHD_NO_HV([
>> +AT_SETUP([NAT on a provider network with no localnet ports])
>> +AT_KEYWORDS([NAT])
>> +ovn_start
>> +
>> +check ovn-nbctl -- ls-add sw0 -- ls-add sw1
>> +check ovn-nbctl lsp-add sw0 sw0-port1
>> +check ovn-nbctl lr-add lr0
>> +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
>> +check ovn-nbctl lsp-add sw0 sw0-lr0
>> +check ovn-nbctl lsp-set-type sw0-lr0 router
>> +check ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01
>> +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
>> +
>> +check ovn-nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:03 20.0.0.1/24
>> +check ovn-nbctl lsp-add sw1 sw1-lr0
>> +check ovn-nbctl lsp-set-type sw1-lr0 router
>> +check ovn-nbctl lsp-set-addresses sw1-lr0 router
>> +check ovn-nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
>> +
>> +check ovn-sbctl chassis-add gw1 geneve 127.0.0.1
>> +check ovn-nbctl ls-add public
>> +check ovn-nbctl lsp-add public pub-p1
>> +
>> +# localnet port
>> +check ovn-nbctl lsp-add public ln-public
>> +check ovn-nbctl lsp-set-type ln-public localnet
>> +check ovn-nbctl lsp-set-addresses ln-public unknown
>> +check ovn-nbctl lsp-set-options ln-public network_name=public
>> +
>> +check ovn-nbctl lrp-add lr0 lr0-public 00:00:00:00:ff:02 172.168.0.10/24
>> +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1
>> +
>> +check ovn-nbctl lsp-add public public-lr0
>> +check ovn-nbctl lsp-set-type public-lr0 router
>> +check ovn-nbctl lsp-set-addresses public-lr0 router
>> +check ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public
>> +
>> +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.3 
>> sw0-port1 30:54:00:00:00:03
>> +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.120 20.0.0.3
>> +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 10.0.0.0/24
>> +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 20.0.0.0/24
>> +
>> +check ovn-nbctl --wait=sb sync
>> +
>> +check_flows_no_cr_port_for_public_lr0() {
>> +  # check that there is no port binding cr-public-lr0
>> +  check_row_count Port_Binding 0 logical_port=cr-public-lr0
>> +
>> +  ovn-sbctl dump-flows lr0 > lr0flows
>> +  ovn-sbctl dump-flows public > publicflows
>> +
>> +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
>> +  table=??(lr_in_admission    ), priority=0    , match=(1), 
>> action=(drop;)
>> +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present 
>> || eth.src[[40]]), action=(drop;)
>> +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && 
>> icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && 
>> icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
>> +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && 
>> icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && 
>> icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && 
>> !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), 
>> action=(outport <-> inport; inport = "lr0-public"; next;)
>> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 
>> 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 
>> 00:00:00:00:ff:01; next;)
>> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 
>> 00:00:00:00:ff:02 && inport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 
>> 00:00:00:00:ff:02; next;)
>> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 
>> 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 
>> 00:00:00:00:ff:03; next;)
>> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 
>> 30:54:00:00:00:03 && inport == "lr0-public" && 
>> is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 
>> 00:00:00:00:ff:02; next;)
>> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && 
>> inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; 
>> next;)
>> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && 
>> inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
>> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && 
>> inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
>> +])
>> +
>> +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
>> +  table=??(lr_in_ip_input     ), priority=0    , match=(1), 
>> action=(next;)
>> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == 
>> {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == 
>> {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == 
>> {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast 
>> ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 
>> 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), 
>> action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == 
>> fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), 
>> action=(reg0 = 0; handle_dhcpv6_reply;)
>> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == 
>> fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), 
>> action=(reg0 = 0; handle_dhcpv6_reply;)
>> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == 
>> fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), 
>> action=(reg0 = 0; handle_dhcpv6_reply;)
>> +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == 
>> "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
>> +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 
>> 1}), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == 
>> "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), 
>> action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded 
>> */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; 
>> ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
>> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == 
>> "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 
>> {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code 
>> = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 
>> 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; 
>> output; };)
>> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == 
>> "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 
>> {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code 
>> = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 
>> 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; 
>> output; };)
>> +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 
>> 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), 
>> action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == 
>> {10.0.0.1}), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == 
>> {172.168.0.10}), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == 
>> {20.0.0.1}), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == 
>> {fe80::200:ff:fe00:ff01}), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == 
>> {fe80::200:ff:fe00:ff02}), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == 
>> {fe80::200:ff:fe00:ff03}), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || 
>> ip6.mcast), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=83   , 
>> match=(ip6.mcast_rsvd), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || 
>> nd_ra), action=(next;)
>> +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), 
>> action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 
>> && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = 
>> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha 
>> = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; 
>> flags.loopback = 1; output;)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 
>> && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = 
>> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha 
>> = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; 
>> flags.loopback = 1; output;)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 
>> && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = 
>> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha 
>> = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; 
>> flags.loopback = 1; output;)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == 
>> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 
>> 172.168.0.0/24 && is_chassis_resident("cr-lr0-public")), 
>> action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* 
>> ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> 
>> arp.spa; outport = inport; flags.loopback = 1; output;)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == 
>> "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} 
>> && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && 
>> is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src 
>> = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; 
>> outport = inport; flags.loopback = 1; output; };)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == 
>> "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 
>> 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; 
>> arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 
>> xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback 
>> = 1; output;)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == 
>> "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && 
>> nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { 
>> eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = 
>> xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == 
>> "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 
>> 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; 
>> arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 
>> xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback 
>> = 1; output;)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == 
>> "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && 
>> nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { 
>> eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = 
>> xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 
>> 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> 
>> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 
>> 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst 
>> <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 
>> 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> 
>> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == 
>> fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), 
>> action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; 
>> flags.loopback = 1; next; )
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == 
>> fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), 
>> action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; 
>> flags.loopback = 1; next; )
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == 
>> fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), 
>> action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; 
>> flags.loopback = 1; next; )
>> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == 
>> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == 
>> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == 
>> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == 
>> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100 && 
>> is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; 
>> eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = 
>> arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = 
>> inport; flags.loopback = 1; output;)
>> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == 
>> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && 
>> is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src 
>> = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; 
>> arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; 
>> flags.loopback = 1; output;)
>> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == 
>> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && 
>> is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; 
>> eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = 
>> arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = 
>> inport; flags.loopback = 1; output;)
>> +])
>> +
>> +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
>> +  table=??(lr_in_unsnat       ), priority=0    , match=(1), 
>> action=(next;)
>> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst 
>> == 172.168.0.100 && inport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
>> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst 
>> == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
>> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst 
>> == 172.168.0.120 && inport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
>> +])
>> +
>> +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
>> +  table=??(lr_in_defrag       ), priority=0    , match=(1), 
>> action=(next;)
>> +])
>> +
>> +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
>> +  table=??(lr_in_dnat         ), priority=0    , match=(1), 
>> action=(next;)
>> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst 
>> == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
>> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst 
>> == 172.168.0.120 && inport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
>> +])
>> +
>> +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], 
>> [dnl
>> +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), 
>> action=(drop;)
>> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), 
>> action=(get_arp(outport, reg0); next;)
>> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), 
>> action=(get_nd(outport, xxreg0); next;)
>> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == 
>> "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 
>> 00:00:00:00:ff:02; next;)
>> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == 
>> "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 
>> 30:54:00:00:00:03; next;)
>> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == 
>> "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 
>> 00:00:00:00:ff:02; next;)
>> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == 
>> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), 
>> action=(drop;)
>> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == 
>> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), 
>> action=(drop;)
>> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == 
>> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), 
>> action=(drop;)
>> +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || 
>> ip6.mcast), action=(next;)
>> +])
>> +
>> +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], 
>> [dnl
>> +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), 
>> action=(next;)
>> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 
>> 10.0.0.3 && outport == "lr0-public" && 
>> is_chassis_resident("sw0-port1")), action=(eth.src = 
>> 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
>> +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == 
>> "lr0-public"), action=(outport = "cr-lr0-public"; next;)
>> +])
>> +
>> +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
>> +  table=??(lr_out_undnat      ), priority=0    , match=(1), 
>> action=(next;)
>> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src 
>> == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 
>> 30:54:00:00:00:03; ct_dnat;)
>> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src 
>> == 20.0.0.3 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
>> +])
>> +
>> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], 
>> [0], [dnl
>> +  table=??(lr_out_post_undnat ), priority=0    , match=(1), 
>> action=(next;)
>> +])
>> +
>> +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
>> +  table=??(lr_out_snat        ), priority=0    , match=(1), 
>> action=(next;)
>> +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), 
>> action=(next;)
>> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src 
>> == 10.0.0.0/24 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), 
>> action=(ct_snat(172.168.0.100);)
>> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src 
>> == 20.0.0.0/24 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), 
>> action=(ct_snat(172.168.0.100);)
>> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src 
>> == 10.0.0.3 && outport == "lr0-public" && 
>> is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), 
>> action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
>> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src 
>> == 20.0.0.3 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), 
>> action=(ct_snat(172.168.0.120);)
>> +])
>> +
>> +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
>> +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), 
>> action=(next;)
>> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 
>> 172.168.0.100 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; 
>> inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; 
>> flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; 
>> reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; 
>> next(pipeline=ingress, table=??); };)
>> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 
>> 172.168.0.110 && outport == "lr0-public" && 
>> is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = 
>> outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback 
>> = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 
>> = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; 
>> next(pipeline=ingress, table=??); };)
>> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 
>> 172.168.0.120 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; 
>> inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; 
>> flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; 
>> reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; 
>> next(pipeline=ingress, table=??); };)
>> +])
>> +
>> +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], 
>> [dnl
>> +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), 
>> action=(outport = get_fdb(eth.dst); next;)
>> +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == 
>> $svc_monitor_mac && (tcp || icmp || icmp6)), 
>> action=(handle_svc_check(inport);)
>> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 
>> 00:00:00:00:ff:02 && is_chassis_resident("cr-lr0-public")), 
>> action=(outport = "public-lr0"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 
>> 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), 
>> action=(outport = "public-lr0"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), 
>> action=(outport = "_MC_flood"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == 
>> {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 
>> || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 
>> 0 && arp.op == 1 && arp.tpa == 172.168.0.10), action=(clone {outport = 
>> "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 
>> 0 && arp.op == 1 && arp.tpa == 172.168.0.100), action=(clone {outport 
>> = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 
>> 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport 
>> = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 
>> 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport 
>> = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 
>> 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02), action=(clone 
>> {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
>> +])
>> +
>> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e 
>> "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | 
>> ovn_strip_lflows], [0], [dnl
>> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 
>> 30:54:00:00:00:03 && inport == "lr0-public" && 
>> is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 
>> 00:00:00:00:ff:02; next;)
>> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == 
>> "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 
>> 30:54:00:00:00:03; next;)
>> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == 
>> "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 
>> 00:00:00:00:ff:02; next;)
>> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == 
>> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), 
>> action=(drop;)
>> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == 
>> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), 
>> action=(drop;)
>> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst 
>> == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
>> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst 
>> == 172.168.0.120 && inport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
>> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 
>> 10.0.0.3 && outport == "lr0-public" && 
>> is_chassis_resident("sw0-port1")), action=(eth.src = 
>> 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 
>> && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = 
>> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha 
>> = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; 
>> flags.loopback = 1; output;)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 
>> && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = 
>> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha 
>> = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; 
>> flags.loopback = 1; output;)
>> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == 
>> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == 
>> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == 
>> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && 
>> is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src 
>> = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; 
>> arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; 
>> flags.loopback = 1; output;)
>> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == 
>> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && 
>> is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; 
>> eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = 
>> arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = 
>> inport; flags.loopback = 1; output;)
>> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst 
>> == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
>> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst 
>> == 172.168.0.120 && inport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
>> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 
>> 172.168.0.110 && outport == "lr0-public" && 
>> is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = 
>> outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback 
>> = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 
>> = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; 
>> next(pipeline=ingress, table=??); };)
>> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 
>> 172.168.0.120 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; 
>> inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; 
>> flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; 
>> reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; 
>> next(pipeline=ingress, table=??); };)
>> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src 
>> == 10.0.0.3 && outport == "lr0-public" && 
>> is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), 
>> action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
>> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src 
>> == 20.0.0.3 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), 
>> action=(ct_snat(172.168.0.120);)
>> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src 
>> == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 
>> 30:54:00:00:00:03; ct_dnat;)
>> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src 
>> == 20.0.0.3 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
>> +])
>> +
>> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e 
>> "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | 
>> ovn_strip_lflows], [0], [dnl
>> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 
>> 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), 
>> action=(outport = "public-lr0"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == 
>> {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 
>> || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 
>> 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport 
>> = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 
>> 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport 
>> = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
>> +])
>> +}
>> +
>> +check_flows_cr_port_for_public_lr0() {
>> +  # check that there is port binding cr-public-lr0
>> +  check_row_count Port_Binding 1 logical_port=cr-public-lr0
>> +  check_column chassisredirect Port_Binding type 
>> logical_port=cr-public-lr0
>> +
>> +  ovn-sbctl dump-flows lr0 > lr0flows
>> +  ovn-sbctl dump-flows public > publicflows
>> +
>> +AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
>> +  table=??(lr_in_admission    ), priority=0    , match=(1), 
>> action=(drop;)
>> +  table=??(lr_in_admission    ), priority=100  , match=(vlan.present 
>> || eth.src[[40]]), action=(drop;)
>> +  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && 
>> icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && 
>> icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
>> +  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && 
>> icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && 
>> icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && 
>> !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), 
>> action=(outport <-> inport; inport = "lr0-public"; next;)
>> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 
>> 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 
>> 00:00:00:00:ff:01; next;)
>> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 
>> 00:00:00:00:ff:02 && inport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 
>> 00:00:00:00:ff:02; next;)
>> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 
>> 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 
>> 00:00:00:00:ff:03; next;)
>> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && 
>> inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; 
>> next;)
>> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && 
>> inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
>> +  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && 
>> inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
>> +])
>> +
>> +AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
>> +  table=??(lr_in_ip_input     ), priority=0    , match=(1), 
>> action=(next;)
>> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == 
>> {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == 
>> {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == 
>> {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast 
>> ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 
>> 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), 
>> action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == 
>> fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), 
>> action=(reg0 = 0; handle_dhcpv6_reply;)
>> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == 
>> fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), 
>> action=(reg0 = 0; handle_dhcpv6_reply;)
>> +  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == 
>> fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), 
>> action=(reg0 = 0; handle_dhcpv6_reply;)
>> +  table=??(lr_in_ip_input     ), priority=120  , match=(inport == 
>> "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
>> +  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 
>> 1}), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == 
>> "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), 
>> action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded 
>> */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; 
>> ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
>> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == 
>> "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 
>> {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code 
>> = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 
>> 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; 
>> output; };)
>> +  table=??(lr_in_ip_input     ), priority=31   , match=(inport == 
>> "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 
>> {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code 
>> = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 
>> 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; 
>> output; };)
>> +  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 
>> 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), 
>> action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == 
>> {10.0.0.1}), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == 
>> {172.168.0.10}), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == 
>> {20.0.0.1}), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == 
>> {fe80::200:ff:fe00:ff01}), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == 
>> {fe80::200:ff:fe00:ff02}), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == 
>> {fe80::200:ff:fe00:ff03}), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || 
>> ip6.mcast), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=83   , 
>> match=(ip6.mcast_rsvd), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || 
>> nd_ra), action=(next;)
>> +  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), 
>> action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 
>> && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = 
>> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha 
>> = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; 
>> flags.loopback = 1; output;)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 
>> && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = 
>> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha 
>> = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; 
>> flags.loopback = 1; output;)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 
>> && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = 
>> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha 
>> = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; 
>> flags.loopback = 1; output;)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == 
>> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 
>> 172.168.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; 
>> arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 
>> xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback 
>> = 1; output;)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == 
>> "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} 
>> && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && 
>> is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src 
>> = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; 
>> outport = inport; flags.loopback = 1; output; };)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == 
>> "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 
>> 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; 
>> arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 
>> xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback 
>> = 1; output;)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == 
>> "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && 
>> nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { 
>> eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = 
>> xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == 
>> "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 
>> 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; 
>> arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 
>> xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback 
>> = 1; output;)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(inport == 
>> "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && 
>> nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { 
>> eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = 
>> xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 
>> 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> 
>> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 
>> 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst 
>> <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 
>> 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> 
>> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == 
>> fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), 
>> action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; 
>> flags.loopback = 1; next; )
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == 
>> fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), 
>> action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; 
>> flags.loopback = 1; next; )
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == 
>> fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), 
>> action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; 
>> flags.loopback = 1; next; )
>> +])
>> +
>> +AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
>> +  table=??(lr_in_unsnat       ), priority=0    , match=(1), 
>> action=(next;)
>> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst 
>> == 172.168.0.100 && inport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
>> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst 
>> == 172.168.0.110 && inport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
>> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst 
>> == 172.168.0.120 && inport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
>> +])
>> +
>> +AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
>> +  table=??(lr_in_defrag       ), priority=0    , match=(1), 
>> action=(next;)
>> +])
>> +
>> +AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
>> +  table=??(lr_in_dnat         ), priority=0    , match=(1), 
>> action=(next;)
>> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst 
>> == 172.168.0.110 && inport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
>> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst 
>> == 172.168.0.120 && inport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
>> +])
>> +
>> +AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], 
>> [dnl
>> +  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), 
>> action=(drop;)
>> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), 
>> action=(get_arp(outport, reg0); next;)
>> +  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), 
>> action=(get_nd(outport, xxreg0); next;)
>> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == 
>> "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 
>> 00:00:00:00:ff:02; next;)
>> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == 
>> "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 
>> 00:00:00:00:ff:02; next;)
>> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == 
>> "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 
>> 00:00:00:00:ff:02; next;)
>> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == 
>> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), 
>> action=(drop;)
>> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == 
>> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), 
>> action=(drop;)
>> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == 
>> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), 
>> action=(drop;)
>> +  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || 
>> ip6.mcast), action=(next;)
>> +])
>> +
>> +AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], 
>> [dnl
>> +  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), 
>> action=(next;)
>> +  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == 
>> "lr0-public"), action=(outport = "cr-lr0-public"; next;)
>> +])
>> +
>> +AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
>> +  table=??(lr_out_undnat      ), priority=0    , match=(1), 
>> action=(next;)
>> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src 
>> == 10.0.0.3 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
>> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src 
>> == 20.0.0.3 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
>> +])
>> +
>> +AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], 
>> [0], [dnl
>> +  table=??(lr_out_post_undnat ), priority=0    , match=(1), 
>> action=(next;)
>> +])
>> +
>> +AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
>> +  table=??(lr_out_snat        ), priority=0    , match=(1), 
>> action=(next;)
>> +  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), 
>> action=(next;)
>> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src 
>> == 10.0.0.0/24 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), 
>> action=(ct_snat(172.168.0.100);)
>> +  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src 
>> == 20.0.0.0/24 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), 
>> action=(ct_snat(172.168.0.100);)
>> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src 
>> == 10.0.0.3 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), 
>> action=(ct_snat(172.168.0.110);)
>> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src 
>> == 20.0.0.3 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), 
>> action=(ct_snat(172.168.0.120);)
>> +])
>> +
>> +AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
>> +  table=??(lr_out_egr_loop    ), priority=0    , match=(1), 
>> action=(next;)
>> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 
>> 172.168.0.100 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; 
>> inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; 
>> flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; 
>> reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; 
>> next(pipeline=ingress, table=??); };)
>> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 
>> 172.168.0.110 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; 
>> inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; 
>> flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; 
>> reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; 
>> next(pipeline=ingress, table=??); };)
>> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 
>> 172.168.0.120 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; 
>> inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; 
>> flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; 
>> reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; 
>> next(pipeline=ingress, table=??); };)
>> +])
>> +
>> +AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], 
>> [dnl
>> +  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), 
>> action=(outport = get_fdb(eth.dst); next;)
>> +  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == 
>> $svc_monitor_mac && (tcp || icmp || icmp6)), 
>> action=(handle_svc_check(inport);)
>> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 
>> 00:00:00:00:ff:02 && !is_chassis_resident("cr-public-lr0")), 
>> action=(outport = "cr-public-lr0"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 
>> 00:00:00:00:ff:02 && is_chassis_resident("cr-public-lr0")), 
>> action=(outport = "public-lr0"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 
>> 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), 
>> action=(outport = "public-lr0"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), 
>> action=(outport = "_MC_flood"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == 
>> {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 
>> || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 
>> 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && 
>> !is_chassis_resident("cr-public-lr0")), action=(clone {outport = 
>> "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 
>> 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && 
>> is_chassis_resident("cr-public-lr0")), action=(clone {outport = 
>> "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 
>> 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && 
>> !is_chassis_resident("cr-public-lr0")), action=(clone {outport = 
>> "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 
>> 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && 
>> is_chassis_resident("cr-public-lr0")), action=(clone {outport = 
>> "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 
>> 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && 
>> !is_chassis_resident("cr-public-lr0")), action=(clone {outport = 
>> "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 
>> 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && 
>> is_chassis_resident("cr-public-lr0")), action=(clone {outport = 
>> "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 
>> 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && 
>> !is_chassis_resident("cr-public-lr0")), action=(clone {outport = 
>> "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 
>> 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && 
>> is_chassis_resident("cr-public-lr0")), action=(clone {outport = 
>> "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 
>> 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && 
>> !is_chassis_resident("cr-public-lr0")), action=(clone {outport = 
>> "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 
>> 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && 
>> is_chassis_resident("cr-public-lr0")), action=(clone {outport = 
>> "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
>> +])
>> +
>> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e 
>> "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | 
>> ovn_strip_lflows], [0], [dnl
>> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == 
>> "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 
>> 00:00:00:00:ff:02; next;)
>> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == 
>> "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 
>> 00:00:00:00:ff:02; next;)
>> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == 
>> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), 
>> action=(drop;)
>> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == 
>> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), 
>> action=(drop;)
>> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst 
>> == 172.168.0.110 && inport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
>> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst 
>> == 172.168.0.120 && inport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 
>> && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = 
>> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha 
>> = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; 
>> flags.loopback = 1; output;)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 
>> && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = 
>> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha 
>> = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; 
>> flags.loopback = 1; output;)
>> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst 
>> == 172.168.0.110 && inport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
>> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst 
>> == 172.168.0.120 && inport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
>> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 
>> 172.168.0.110 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; 
>> inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; 
>> flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; 
>> reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; 
>> next(pipeline=ingress, table=??); };)
>> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 
>> 172.168.0.120 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; 
>> inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; 
>> flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; 
>> reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; 
>> next(pipeline=ingress, table=??); };)
>> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src 
>> == 10.0.0.3 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), 
>> action=(ct_snat(172.168.0.110);)
>> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src 
>> == 20.0.0.3 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), 
>> action=(ct_snat(172.168.0.120);)
>> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src 
>> == 10.0.0.3 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
>> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src 
>> == 20.0.0.3 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
>> +])
>> +
>> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e 
>> "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | 
>> ovn_strip_lflows], [0], [dnl
>> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 
>> 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), 
>> action=(outport = "public-lr0"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == 
>> {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 
>> || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 
>> 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && 
>> !is_chassis_resident("cr-public-lr0")), action=(clone {outport = 
>> "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 
>> 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && 
>> is_chassis_resident("cr-public-lr0")), action=(clone {outport = 
>> "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 
>> 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && 
>> !is_chassis_resident("cr-public-lr0")), action=(clone {outport = 
>> "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 
>> 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && 
>> is_chassis_resident("cr-public-lr0")), action=(clone {outport = 
>> "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
>> +])
>> +}
>> +
>> +# Check that the lflows are as expected when public has localnet port.
>> +check_flows_no_cr_port_for_public_lr0
>> +
>> +# Remove the localnet port from public logical switch.
>> +check ovn-nbctl --wait=sb lsp-set-type ln-public ""
>> +
>> +# Check that the lflows are as expected and there is no cr port
>> +# created for "public-lr0"  when public has no localnet port
>> +# since public doesn't have the option "overlay_provider_network=true"
>> +# set.
>> +check_row_count Port_Binding 0 logical_port=cr-public-lr0
>> +
>> +ovn-sbctl dump-flows lr0 > lr0flows
>> +ovn-sbctl dump-flows public > publicflows
>> +
>> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e 
>> "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | 
>> ovn_strip_lflows], [0], [dnl
>> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 
>> 30:54:00:00:00:03 && inport == "lr0-public" && 
>> is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 
>> 00:00:00:00:ff:02; next;)
>> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == 
>> "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 
>> 30:54:00:00:00:03; next;)
>> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == 
>> "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 
>> 00:00:00:00:ff:02; next;)
>> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == 
>> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), 
>> action=(drop;)
>> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == 
>> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), 
>> action=(drop;)
>> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst 
>> == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
>> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst 
>> == 172.168.0.120 && inport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
>> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 
>> 10.0.0.3 && outport == "lr0-public" && 
>> is_chassis_resident("sw0-port1")), action=(eth.src = 
>> 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 
>> && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = 
>> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha 
>> = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; 
>> flags.loopback = 1; output;)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 
>> && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = 
>> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha 
>> = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; 
>> flags.loopback = 1; output;)
>> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == 
>> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == 
>> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == 
>> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && 
>> is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src 
>> = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; 
>> arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; 
>> flags.loopback = 1; output;)
>> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == 
>> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && 
>> is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; 
>> eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = 
>> arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = 
>> inport; flags.loopback = 1; output;)
>> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst 
>> == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
>> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst 
>> == 172.168.0.120 && inport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
>> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 
>> 172.168.0.110 && outport == "lr0-public" && 
>> is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = 
>> outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback 
>> = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 
>> = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; 
>> next(pipeline=ingress, table=??); };)
>> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 
>> 172.168.0.120 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; 
>> inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; 
>> flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; 
>> reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; 
>> next(pipeline=ingress, table=??); };)
>> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src 
>> == 10.0.0.3 && outport == "lr0-public" && 
>> is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), 
>> action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
>> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src 
>> == 20.0.0.3 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), 
>> action=(ct_snat(172.168.0.120);)
>> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src 
>> == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 
>> 30:54:00:00:00:03; ct_dnat;)
>> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src 
>> == 20.0.0.3 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
>> +])
>> +
>> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e 
>> "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | 
>> ovn_strip_lflows], [0], [dnl
>> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 
>> 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), 
>> action=(outport = "public-lr0"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == 
>> {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 
>> || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 
>> 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport 
>> = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 
>> 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport 
>> = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
>> +])
>> +
>> +
>> +# Set the option "centralize_routing=true" for lr0-public.
>> +check ovn-nbctl --wait=sb set logical_router_port lr0-public 
>> options:centralize_routing=true
>> +
>> +# Check that the lflows are as expected and there is cr port created 
>> for public-lr0.
>> +check_flows_cr_port_for_public_lr0
>> +
>> +# Set the type of ln-public back to localnet
>> +check ovn-nbctl --wait=sb lsp-set-type ln-public localnet
>> +
>> +# Check that the lflows are as expected when public has localnet port.
>> +check_flows_no_cr_port_for_public_lr0
>> +
>> +# Delete the localnet port
>> +check ovn-nbctl --wait=sb lsp-del ln-public
>> +
>> +# Check that the lflows are as expected when public has no localnet 
>> port.
>> +check_flows_cr_port_for_public_lr0
>> +
>> +# Create multiple gateway ports.  chassisresident port should not be
>> +# created for 'public-lr0' even if there is no localnet port on 'public'
>> +# logical switch.
>> +check ovn-nbctl --wait=sb lrp-set-gateway-chassis lr0-sw0 gw1
>> +# check that there is no port binding cr-public-lr0
>> +check_row_count Port_Binding 0 logical_port=cr-public-lr0
>> +
>> +ovn-sbctl dump-flows lr0 > lr0flows
>> +ovn-sbctl dump-flows public > publicflows
>> +
>> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e 
>> "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | 
>> ovn_strip_lflows], [0], [dnl
>> +  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 
>> 30:54:00:00:00:03 && inport == "lr0-public" && 
>> is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 
>> 00:00:00:00:ff:02; next;)
>> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == 
>> "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 
>> 30:54:00:00:00:03; next;)
>> +  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == 
>> "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 
>> 00:00:00:00:ff:02; next;)
>> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == 
>> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), 
>> action=(drop;)
>> +  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == 
>> "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), 
>> action=(drop;)
>> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst 
>> == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
>> +  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst 
>> == 172.168.0.120 && inport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
>> +  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 
>> 10.0.0.3 && outport == "lr0-public" && 
>> is_chassis_resident("sw0-port1")), action=(eth.src = 
>> 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 
>> && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = 
>> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha 
>> = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; 
>> flags.loopback = 1; output;)
>> +  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 
>> && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = 
>> xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha 
>> = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; 
>> flags.loopback = 1; output;)
>> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == 
>> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=91   , match=(inport == 
>> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
>> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == 
>> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && 
>> is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src 
>> = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; 
>> arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; 
>> flags.loopback = 1; output;)
>> +  table=??(lr_in_ip_input     ), priority=92   , match=(inport == 
>> "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && 
>> is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; 
>> eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = 
>> arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = 
>> inport; flags.loopback = 1; output;)
>> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst 
>> == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
>> +  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst 
>> == 172.168.0.120 && inport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
>> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 
>> 172.168.0.110 && outport == "lr0-public" && 
>> is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = 
>> outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback 
>> = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 
>> = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; 
>> next(pipeline=ingress, table=??); };)
>> +  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 
>> 172.168.0.120 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; 
>> inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; 
>> flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; 
>> reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; 
>> next(pipeline=ingress, table=??); };)
>> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src 
>> == 10.0.0.3 && outport == "lr0-public" && 
>> is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), 
>> action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
>> +  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src 
>> == 20.0.0.3 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), 
>> action=(ct_snat(172.168.0.120);)
>> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src 
>> == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 
>> 30:54:00:00:00:03; ct_dnat;)
>> +  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src 
>> == 20.0.0.3 && outport == "lr0-public" && 
>> is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
>> +])
>> +
>> +AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e 
>> "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | 
>> ovn_strip_lflows], [0], [dnl
>> +  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 
>> 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), 
>> action=(outport = "public-lr0"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == 
>> {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 
>> || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 
>> 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport 
>> = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
>> +  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 
>> 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport 
>> = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
>> +])
>> +
>> +AT_CLEANUP
>> +])
>> diff --git a/tests/ovn.at b/tests/ovn.at
>> index 582ec56485..c468673f0c 100644
>> --- a/tests/ovn.at
>> +++ b/tests/ovn.at
>> @@ -21227,10 +21227,10 @@ ovn-nbctl lsp-add sw0 rp-sw0 -- set 
>> Logical_Switch_Port rp-sw0 \
>>       type=router options:router-port=sw0 \
>>       -- lsp-set-addresses rp-sw0 router
>> -ovn-nbctl lrp-add lr0 sw1 00:00:02:01:02:03 172.16.1.1/24 
>> 2002:0:0:0:0:0:0:1/64 \
>> -    -- lrp-set-gateway-chassis sw1 hv2
>> +ovn-nbctl lrp-add lr0 lr0-sw1 00:00:02:01:02:03 172.16.1.1/24 
>> 2002:0:0:0:0:0:0:1/64 \
>> +    -- lrp-set-gateway-chassis lr0-sw1 hv2
>>   ovn-nbctl lsp-add sw1 rp-sw1 -- set Logical_Switch_Port rp-sw1 \
>> -    type=router options:router-port=sw1 \
>> +    type=router options:router-port=lr0-sw1 \
>>       -- lsp-set-addresses rp-sw1 router
>>   ovn-nbctl lsp-add sw0 sw0-p0 \
>> @@ -21242,6 +21242,8 @@ ovn-nbctl lsp-add sw0 sw0-p1 \
>>   ovn-nbctl lsp-add sw1 sw1-p0 \
>>       -- lsp-set-addresses sw1-p0 unknown
>> +check ovn-nbctl lsp-add sw1 ln-sw1 -- lsp-set-type ln-sw1 localnet
>> +
>>   ovn-nbctl lr-nat-add lr0 snat 172.16.1.1 192.168.1.0/24
>>   ovn-nbctl lr-nat-add lr0 snat 2002::1 2001::/64
>
Numan Siddique Aug. 7, 2024, 6:30 p.m. UTC | #10 
On Wed, Aug 7, 2024 at 11:35 AM Han Zhou <hzhou@ovn.org> wrote:
>
> On Wed, Aug 7, 2024 at 7:18 AM Felix Huettner
> <felix.huettner@mail.schwarz> wrote:
> >
> > On Wed, Aug 07, 2024 at 12:11:42AM -0700, Han Zhou wrote:
> > > [Einige Personen, die diese Nachricht erhalten haben, erhalten häufig keine E-Mails von zhouhan@gmail.com. Weitere Informationen, warum dies wichtig ist, finden Sie unter https://aka.ms/LearnAboutSenderIdentification ]
> > >
> > > On Tue, Aug 6, 2024 at 8:35 PM Numan Siddique <numans@ovn.org> wrote:
> > > >
> > > > On Tue, Aug 6, 2024 at 2:50 AM Han Zhou <zhouhan@gmail.com> wrote:
> > > > >
> > > > > On Mon, Jul 29, 2024 at 7:39 PM <numans@ovn.org> wrote:
> > > > > >
> > > > > > From: Numan Siddique <numans@ovn.org>
> > > > > >
> > > > > > Consider a deployment with the below logical resources:
> > > > > >
> > > > > > 1. A bridged logical switch 'public' with a port - P1 and a localnet
> > > > > >    port ln-public.
> > > > > > 2. A logical router 'R'
> > > > > > 3. Logical switch 'public' connected to R via logical switch/router port
> > > > > >    peers (public-R and R-public).
> > > > > > 4. R-public is distributed gateway port with its network as 172.16.0.0/24
> > > > > > 5. NATs (dnat_and_snat) configured in 'R'.
> > > > > > 6. And a few overlay logical switches S1, S2 to R.
> > > > > >
> > > > > > Any traffic from logical port - P1 of public logical switch destined to
> > > > > > S1 or S2's logical ports goes out of the source chassis
> > > > > > (where P1 resides) via the localnet port and reaches the gateway chassis
> > > > > > which handles the routing.
> > > > > >
> > > > > > There are couple of traffic flow scenarios which doesn't work if the
> > > > > > logical switch 'public' doesn't have a localnet port.
> > > > > >
> > > > > > 1. Traffic from port - P1 destined to logical switches S1 or S2 gets
> > > > > >    dropped in the source chassis.  The packet enters the router R's
> > > > > >    pipeline, but it gets dropped in the 'lr_in_admission' stage since
> > > > > >    the logical flow to allow traffic destined to the distributed gateway
> > > > > >    port MAC is installed only on the gateway chassis.
> > > > > >
> > > > > > 2. NAT doesn't work as expected.
> > > > > >
> > > > > > In order to suppose this use case (of a logical switch not having a
> > > > > > localnet port, but has a distributed gateway port and NATs), this patch
> > > > > > supports the option 'centralize_routing', which can be configured on
> > > > > > the distributed gateway port (R-public in the example above).
> > > > > > If this option is set, then routing is centralized on the gateway
> > > > > > chassis for the traffic destined to the R-public's networks
> > > > > > (172.16.0.0/24 for the above example).  Traffic from P1 will be
> > > > > > tunnelled to the gateway chassis.
> > > > > >
> > > > > > ovn-northd creates a chassisresident port (cr-public-R) for the
> > > > > > logical switch port - public-R, along with cr-R-public inorder to
> > > > > > centralize the traffic.
> > > > > >
> > > > > Hi Numan, I have two questions here:
> > > > >
> > > > > > This feature gets enabled for the distributed gateway port R-public if
> > > > > >   - The above option is set to true in the R-public's options column.
> > > > >
> > > > > Why do we need an option to specify this behavior? If the LRP is a
> > > > > distributed gateway port and there is no localnet port on the
> > > > > connected LS, can't we just do the centralized routing?
> > > >
> > > > That was my initial approach.  But many test cases fail [1] with this approach.
> > > > It was hard to fix all these test cases and I was not sure if I'd
> > > > break any existing
> > > > scenarios.
> > > >
> > > > [1] https://github.com/numansiddique/ovn/actions/runs/10275831079/job/28435316245
> > > > testsuite: 75 76 161 162 163 164 165 166 167 168 169 170 183 184 185
> > > > 186 187 188 271 272 291 292 failed
> > > >
> > >
> > > I am a little concerned about this. The original behavior is like a
> > > bug, because a LSP on the external overlay network is supposed to be
> > > able to communicate with the LSPs in the internal overlay networks. So
> > > I understand that this patch is trying to fix that. However, the fix
> > > would break some scenarios and it is not very clear why they break
> > > what scenarios would break. Because of this you added a new option to
> > > workaround the problem, because the test cases that didn't enable this
> > > option would still pass for sure. However, it also means that when
> > > this option is enabled, we are not very sure what other scenarios
> > > would break. Would it be better to examine the failed cases to
> > > understand what is really broken?
> >
> > From my usage and tests all of these should be fixed by
> > https://mail.openvswitch.org/pipermail/ovs-dev/2024-August/416264.html
> >
> > Then i guess the option can be removed as well.
> >
>
> Thanks Felix. Hope this solves the problem.
>
> Numan, I don't have any other comment to the code except that the word
> "chassisresident" in code (and also in the commit message) may be
> confusing. We have been using the term chassisredirect or
> chassis-redirect which is what "CR" stands for. I see only one
> occurrence of chassisresident which is in the comment of is_cr_port()
> and I believe that was a typo. So would be better to unify the term
> and stick to chassisredirect in the var namings and comments.
>
> I will wait for v3 and like to do some tests myself.

Looks like @Mark Michelson  didn't follow the discussion here and
merged the patch to main.

I'll submit another patch to fix the crash reported by @Felix Hüttner
and also address the "chassisresident" comment
from @Han Zhou .

@Han Zhou   - Regarding the option,  I'm a little concerned about
enabling this behavior by default.
If we enable this patch's behavior by default, then we will create a
chassis resident port for
the  gateway router port's peer.  By looking into the number of test
failures,  I was a bit concerned that
this may break any existing deployments and hence added the option.

We can definitely debug the test failures and make this option true by
default, post 24.09.
I'll look into that post 24.09 release (also a little short of cycles
as I want to focus on some reviews).

Thanks
Numan


>
> Thanks,
> Han
>
> > Thanks
> > Felix
> >
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Han Zhou Aug. 7, 2024, 7:52 p.m. UTC | #11 
On Wed, Aug 7, 2024 at 11:30 AM Numan Siddique <numans@ovn.org> wrote:
>
> On Wed, Aug 7, 2024 at 11:35 AM Han Zhou <hzhou@ovn.org> wrote:
> >
> > On Wed, Aug 7, 2024 at 7:18 AM Felix Huettner
> > <felix.huettner@mail.schwarz> wrote:
> > >
> > > On Wed, Aug 07, 2024 at 12:11:42AM -0700, Han Zhou wrote:
> > > > [Einige Personen, die diese Nachricht erhalten haben, erhalten häufig keine E-Mails von zhouhan@gmail.com. Weitere Informationen, warum dies wichtig ist, finden Sie unter https://aka.ms/LearnAboutSenderIdentification ]
> > > >
> > > > On Tue, Aug 6, 2024 at 8:35 PM Numan Siddique <numans@ovn.org> wrote:
> > > > >
> > > > > On Tue, Aug 6, 2024 at 2:50 AM Han Zhou <zhouhan@gmail.com> wrote:
> > > > > >
> > > > > > On Mon, Jul 29, 2024 at 7:39 PM <numans@ovn.org> wrote:
> > > > > > >
> > > > > > > From: Numan Siddique <numans@ovn.org>
> > > > > > >
> > > > > > > Consider a deployment with the below logical resources:
> > > > > > >
> > > > > > > 1. A bridged logical switch 'public' with a port - P1 and a localnet
> > > > > > >    port ln-public.
> > > > > > > 2. A logical router 'R'
> > > > > > > 3. Logical switch 'public' connected to R via logical switch/router port
> > > > > > >    peers (public-R and R-public).
> > > > > > > 4. R-public is distributed gateway port with its network as 172.16.0.0/24
> > > > > > > 5. NATs (dnat_and_snat) configured in 'R'.
> > > > > > > 6. And a few overlay logical switches S1, S2 to R.
> > > > > > >
> > > > > > > Any traffic from logical port - P1 of public logical switch destined to
> > > > > > > S1 or S2's logical ports goes out of the source chassis
> > > > > > > (where P1 resides) via the localnet port and reaches the gateway chassis
> > > > > > > which handles the routing.
> > > > > > >
> > > > > > > There are couple of traffic flow scenarios which doesn't work if the
> > > > > > > logical switch 'public' doesn't have a localnet port.
> > > > > > >
> > > > > > > 1. Traffic from port - P1 destined to logical switches S1 or S2 gets
> > > > > > >    dropped in the source chassis.  The packet enters the router R's
> > > > > > >    pipeline, but it gets dropped in the 'lr_in_admission' stage since
> > > > > > >    the logical flow to allow traffic destined to the distributed gateway
> > > > > > >    port MAC is installed only on the gateway chassis.
> > > > > > >
> > > > > > > 2. NAT doesn't work as expected.
> > > > > > >
> > > > > > > In order to suppose this use case (of a logical switch not having a
> > > > > > > localnet port, but has a distributed gateway port and NATs), this patch
> > > > > > > supports the option 'centralize_routing', which can be configured on
> > > > > > > the distributed gateway port (R-public in the example above).
> > > > > > > If this option is set, then routing is centralized on the gateway
> > > > > > > chassis for the traffic destined to the R-public's networks
> > > > > > > (172.16.0.0/24 for the above example).  Traffic from P1 will be
> > > > > > > tunnelled to the gateway chassis.
> > > > > > >
> > > > > > > ovn-northd creates a chassisresident port (cr-public-R) for the
> > > > > > > logical switch port - public-R, along with cr-R-public inorder to
> > > > > > > centralize the traffic.
> > > > > > >
> > > > > > Hi Numan, I have two questions here:
> > > > > >
> > > > > > > This feature gets enabled for the distributed gateway port R-public if
> > > > > > >   - The above option is set to true in the R-public's options column.
> > > > > >
> > > > > > Why do we need an option to specify this behavior? If the LRP is a
> > > > > > distributed gateway port and there is no localnet port on the
> > > > > > connected LS, can't we just do the centralized routing?
> > > > >
> > > > > That was my initial approach.  But many test cases fail [1] with this approach.
> > > > > It was hard to fix all these test cases and I was not sure if I'd
> > > > > break any existing
> > > > > scenarios.
> > > > >
> > > > > [1] https://github.com/numansiddique/ovn/actions/runs/10275831079/job/28435316245
> > > > > testsuite: 75 76 161 162 163 164 165 166 167 168 169 170 183 184 185
> > > > > 186 187 188 271 272 291 292 failed
> > > > >
> > > >
> > > > I am a little concerned about this. The original behavior is like a
> > > > bug, because a LSP on the external overlay network is supposed to be
> > > > able to communicate with the LSPs in the internal overlay networks. So
> > > > I understand that this patch is trying to fix that. However, the fix
> > > > would break some scenarios and it is not very clear why they break
> > > > what scenarios would break. Because of this you added a new option to
> > > > workaround the problem, because the test cases that didn't enable this
> > > > option would still pass for sure. However, it also means that when
> > > > this option is enabled, we are not very sure what other scenarios
> > > > would break. Would it be better to examine the failed cases to
> > > > understand what is really broken?
> > >
> > > From my usage and tests all of these should be fixed by
> > > https://mail.openvswitch.org/pipermail/ovs-dev/2024-August/416264.html
> > >
> > > Then i guess the option can be removed as well.
> > >
> >
> > Thanks Felix. Hope this solves the problem.
> >
> > Numan, I don't have any other comment to the code except that the word
> > "chassisresident" in code (and also in the commit message) may be
> > confusing. We have been using the term chassisredirect or
> > chassis-redirect which is what "CR" stands for. I see only one
> > occurrence of chassisresident which is in the comment of is_cr_port()
> > and I believe that was a typo. So would be better to unify the term
> > and stick to chassisredirect in the var namings and comments.
> >
> > I will wait for v3 and like to do some tests myself.
>
> Looks like @Mark Michelson  didn't follow the discussion here and
> merged the patch to main.
>
oops :D

> I'll submit another patch to fix the crash reported by @Felix Hüttner
> and also address the "chassisresident" comment
> from @Han Zhou .
>
> @Han Zhou   - Regarding the option,  I'm a little concerned about
> enabling this behavior by default.
> If we enable this patch's behavior by default, then we will create a
> chassis resident port for
> the  gateway router port's peer.  By looking into the number of test
> failures,  I was a bit concerned that
> this may break any existing deployments and hence added the option.
>

Ok, I think of it as a bug fix but it sounds like an experimental
feature. From the code it looks ok to me to create a chassis-redirect
port for DGP's peer. I understand this may be risky because the change
seems big. Introducing an option reduces that risk, but we need to
keep in mind that for the use cases that require
centralize_routing=true, the risk still exists, unless we figure out
all the failed cases and we are sure those failed scenarios are not
required for the centralize_routing=true use cases. At that time I'd
rather remove the option instead of just changing the default to true,
to avoid the burden of maintaining such an option if it SHOULD be true
from a design point of view.

Thanks,
Han

> We can definitely debug the test failures and make this option true by
> default, post 24.09.
> I'll look into that post 24.09 release (also a little short of cycles
> as I want to focus on some reviews).
>
> Thanks
> Numan
>
>
> >
> > Thanks,
> > Han
> >
> > > Thanks
> > > Felix
> > >
> > _______________________________________________
> > dev mailing list
> > dev@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Numan Siddique Aug. 7, 2024, 7:58 p.m. UTC | #12 
On Wed, Aug 7, 2024 at 3:53 PM Han Zhou <hzhou@ovn.org> wrote:
>
> On Wed, Aug 7, 2024 at 11:30 AM Numan Siddique <numans@ovn.org> wrote:
> >
> > On Wed, Aug 7, 2024 at 11:35 AM Han Zhou <hzhou@ovn.org> wrote:
> > >
> > > On Wed, Aug 7, 2024 at 7:18 AM Felix Huettner
> > > <felix.huettner@mail.schwarz> wrote:
> > > >
> > > > On Wed, Aug 07, 2024 at 12:11:42AM -0700, Han Zhou wrote:
> > > > > [Einige Personen, die diese Nachricht erhalten haben, erhalten häufig keine E-Mails von zhouhan@gmail.com. Weitere Informationen, warum dies wichtig ist, finden Sie unter https://aka.ms/LearnAboutSenderIdentification ]
> > > > >
> > > > > On Tue, Aug 6, 2024 at 8:35 PM Numan Siddique <numans@ovn.org> wrote:
> > > > > >
> > > > > > On Tue, Aug 6, 2024 at 2:50 AM Han Zhou <zhouhan@gmail.com> wrote:
> > > > > > >
> > > > > > > On Mon, Jul 29, 2024 at 7:39 PM <numans@ovn.org> wrote:
> > > > > > > >
> > > > > > > > From: Numan Siddique <numans@ovn.org>
> > > > > > > >
> > > > > > > > Consider a deployment with the below logical resources:
> > > > > > > >
> > > > > > > > 1. A bridged logical switch 'public' with a port - P1 and a localnet
> > > > > > > >    port ln-public.
> > > > > > > > 2. A logical router 'R'
> > > > > > > > 3. Logical switch 'public' connected to R via logical switch/router port
> > > > > > > >    peers (public-R and R-public).
> > > > > > > > 4. R-public is distributed gateway port with its network as 172.16.0.0/24
> > > > > > > > 5. NATs (dnat_and_snat) configured in 'R'.
> > > > > > > > 6. And a few overlay logical switches S1, S2 to R.
> > > > > > > >
> > > > > > > > Any traffic from logical port - P1 of public logical switch destined to
> > > > > > > > S1 or S2's logical ports goes out of the source chassis
> > > > > > > > (where P1 resides) via the localnet port and reaches the gateway chassis
> > > > > > > > which handles the routing.
> > > > > > > >
> > > > > > > > There are couple of traffic flow scenarios which doesn't work if the
> > > > > > > > logical switch 'public' doesn't have a localnet port.
> > > > > > > >
> > > > > > > > 1. Traffic from port - P1 destined to logical switches S1 or S2 gets
> > > > > > > >    dropped in the source chassis.  The packet enters the router R's
> > > > > > > >    pipeline, but it gets dropped in the 'lr_in_admission' stage since
> > > > > > > >    the logical flow to allow traffic destined to the distributed gateway
> > > > > > > >    port MAC is installed only on the gateway chassis.
> > > > > > > >
> > > > > > > > 2. NAT doesn't work as expected.
> > > > > > > >
> > > > > > > > In order to suppose this use case (of a logical switch not having a
> > > > > > > > localnet port, but has a distributed gateway port and NATs), this patch
> > > > > > > > supports the option 'centralize_routing', which can be configured on
> > > > > > > > the distributed gateway port (R-public in the example above).
> > > > > > > > If this option is set, then routing is centralized on the gateway
> > > > > > > > chassis for the traffic destined to the R-public's networks
> > > > > > > > (172.16.0.0/24 for the above example).  Traffic from P1 will be
> > > > > > > > tunnelled to the gateway chassis.
> > > > > > > >
> > > > > > > > ovn-northd creates a chassisresident port (cr-public-R) for the
> > > > > > > > logical switch port - public-R, along with cr-R-public inorder to
> > > > > > > > centralize the traffic.
> > > > > > > >
> > > > > > > Hi Numan, I have two questions here:
> > > > > > >
> > > > > > > > This feature gets enabled for the distributed gateway port R-public if
> > > > > > > >   - The above option is set to true in the R-public's options column.
> > > > > > >
> > > > > > > Why do we need an option to specify this behavior? If the LRP is a
> > > > > > > distributed gateway port and there is no localnet port on the
> > > > > > > connected LS, can't we just do the centralized routing?
> > > > > >
> > > > > > That was my initial approach.  But many test cases fail [1] with this approach.
> > > > > > It was hard to fix all these test cases and I was not sure if I'd
> > > > > > break any existing
> > > > > > scenarios.
> > > > > >
> > > > > > [1] https://github.com/numansiddique/ovn/actions/runs/10275831079/job/28435316245
> > > > > > testsuite: 75 76 161 162 163 164 165 166 167 168 169 170 183 184 185
> > > > > > 186 187 188 271 272 291 292 failed
> > > > > >
> > > > >
> > > > > I am a little concerned about this. The original behavior is like a
> > > > > bug, because a LSP on the external overlay network is supposed to be
> > > > > able to communicate with the LSPs in the internal overlay networks. So
> > > > > I understand that this patch is trying to fix that. However, the fix
> > > > > would break some scenarios and it is not very clear why they break
> > > > > what scenarios would break. Because of this you added a new option to
> > > > > workaround the problem, because the test cases that didn't enable this
> > > > > option would still pass for sure. However, it also means that when
> > > > > this option is enabled, we are not very sure what other scenarios
> > > > > would break. Would it be better to examine the failed cases to
> > > > > understand what is really broken?
> > > >
> > > > From my usage and tests all of these should be fixed by
> > > > https://mail.openvswitch.org/pipermail/ovs-dev/2024-August/416264.html
> > > >
> > > > Then i guess the option can be removed as well.
> > > >
> > >
> > > Thanks Felix. Hope this solves the problem.
> > >
> > > Numan, I don't have any other comment to the code except that the word
> > > "chassisresident" in code (and also in the commit message) may be
> > > confusing. We have been using the term chassisredirect or
> > > chassis-redirect which is what "CR" stands for. I see only one
> > > occurrence of chassisresident which is in the comment of is_cr_port()
> > > and I believe that was a typo. So would be better to unify the term
> > > and stick to chassisredirect in the var namings and comments.
> > >
> > > I will wait for v3 and like to do some tests myself.
> >
> > Looks like @Mark Michelson  didn't follow the discussion here and
> > merged the patch to main.
> >
> oops :D
>
> > I'll submit another patch to fix the crash reported by @Felix Hüttner
> > and also address the "chassisresident" comment
> > from @Han Zhou .
> >
> > @Han Zhou   - Regarding the option,  I'm a little concerned about
> > enabling this behavior by default.
> > If we enable this patch's behavior by default, then we will create a
> > chassis resident port for
> > the  gateway router port's peer.  By looking into the number of test
> > failures,  I was a bit concerned that
> > this may break any existing deployments and hence added the option.
> >
>
> Ok, I think of it as a bug fix but it sounds like an experimental
> feature. From the code it looks ok to me to create a chassis-redirect
> port for DGP's peer. I understand this may be risky because the change
> seems big. Introducing an option reduces that risk, but we need to
> keep in mind that for the use cases that require
> centralize_routing=true, the risk still exists, unless we figure out
> all the failed cases and we are sure those failed scenarios are not
> required for the centralize_routing=true use cases. At that time I'd
> rather remove the option instead of just changing the default to true,
> to avoid the burden of maintaining such an option if it SHOULD be true
> from a design point of view.

Sounds good to me.

How about I submit another patch to mark it as experimental once this patch
is merged - https://patchwork.ozlabs.org/project/ovn/patch/20240802155449.3528777-1-mmichels@redhat.com/
 ?
And later remove the option altogether after addressing all the failed
scenarios ?

Numan

>
> Thanks,
> Han
>
> > We can definitely debug the test failures and make this option true by
> > default, post 24.09.
> > I'll look into that post 24.09 release (also a little short of cycles
> > as I want to focus on some reviews).
> >
> > Thanks
> > Numan
> >
> >
> > >
> > > Thanks,
> > > Han
> > >
> > > > Thanks
> > > > Felix
> > > >
> > > _______________________________________________
> > > dev mailing list
> > > dev@openvswitch.org
> > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Han Zhou Aug. 7, 2024, 9:39 p.m. UTC | #13 
On Wed, Aug 7, 2024 at 12:58 PM Numan Siddique <numans@ovn.org> wrote:
>
> On Wed, Aug 7, 2024 at 3:53 PM Han Zhou <hzhou@ovn.org> wrote:
> >
> > On Wed, Aug 7, 2024 at 11:30 AM Numan Siddique <numans@ovn.org> wrote:
> > >
> > > On Wed, Aug 7, 2024 at 11:35 AM Han Zhou <hzhou@ovn.org> wrote:
> > > >
> > > > On Wed, Aug 7, 2024 at 7:18 AM Felix Huettner
> > > > <felix.huettner@mail.schwarz> wrote:
> > > > >
> > > > > On Wed, Aug 07, 2024 at 12:11:42AM -0700, Han Zhou wrote:
> > > > > > [Einige Personen, die diese Nachricht erhalten haben, erhalten häufig keine E-Mails von zhouhan@gmail.com. Weitere Informationen, warum dies wichtig ist, finden Sie unter https://aka.ms/LearnAboutSenderIdentification ]
> > > > > >
> > > > > > On Tue, Aug 6, 2024 at 8:35 PM Numan Siddique <numans@ovn.org> wrote:
> > > > > > >
> > > > > > > On Tue, Aug 6, 2024 at 2:50 AM Han Zhou <zhouhan@gmail.com> wrote:
> > > > > > > >
> > > > > > > > On Mon, Jul 29, 2024 at 7:39 PM <numans@ovn.org> wrote:
> > > > > > > > >
> > > > > > > > > From: Numan Siddique <numans@ovn.org>
> > > > > > > > >
> > > > > > > > > Consider a deployment with the below logical resources:
> > > > > > > > >
> > > > > > > > > 1. A bridged logical switch 'public' with a port - P1 and a localnet
> > > > > > > > >    port ln-public.
> > > > > > > > > 2. A logical router 'R'
> > > > > > > > > 3. Logical switch 'public' connected to R via logical switch/router port
> > > > > > > > >    peers (public-R and R-public).
> > > > > > > > > 4. R-public is distributed gateway port with its network as 172.16.0.0/24
> > > > > > > > > 5. NATs (dnat_and_snat) configured in 'R'.
> > > > > > > > > 6. And a few overlay logical switches S1, S2 to R.
> > > > > > > > >
> > > > > > > > > Any traffic from logical port - P1 of public logical switch destined to
> > > > > > > > > S1 or S2's logical ports goes out of the source chassis
> > > > > > > > > (where P1 resides) via the localnet port and reaches the gateway chassis
> > > > > > > > > which handles the routing.
> > > > > > > > >
> > > > > > > > > There are couple of traffic flow scenarios which doesn't work if the
> > > > > > > > > logical switch 'public' doesn't have a localnet port.
> > > > > > > > >
> > > > > > > > > 1. Traffic from port - P1 destined to logical switches S1 or S2 gets
> > > > > > > > >    dropped in the source chassis.  The packet enters the router R's
> > > > > > > > >    pipeline, but it gets dropped in the 'lr_in_admission' stage since
> > > > > > > > >    the logical flow to allow traffic destined to the distributed gateway
> > > > > > > > >    port MAC is installed only on the gateway chassis.
> > > > > > > > >
> > > > > > > > > 2. NAT doesn't work as expected.
> > > > > > > > >
> > > > > > > > > In order to suppose this use case (of a logical switch not having a
> > > > > > > > > localnet port, but has a distributed gateway port and NATs), this patch
> > > > > > > > > supports the option 'centralize_routing', which can be configured on
> > > > > > > > > the distributed gateway port (R-public in the example above).
> > > > > > > > > If this option is set, then routing is centralized on the gateway
> > > > > > > > > chassis for the traffic destined to the R-public's networks
> > > > > > > > > (172.16.0.0/24 for the above example).  Traffic from P1 will be
> > > > > > > > > tunnelled to the gateway chassis.
> > > > > > > > >
> > > > > > > > > ovn-northd creates a chassisresident port (cr-public-R) for the
> > > > > > > > > logical switch port - public-R, along with cr-R-public inorder to
> > > > > > > > > centralize the traffic.
> > > > > > > > >
> > > > > > > > Hi Numan, I have two questions here:
> > > > > > > >
> > > > > > > > > This feature gets enabled for the distributed gateway port R-public if
> > > > > > > > >   - The above option is set to true in the R-public's options column.
> > > > > > > >
> > > > > > > > Why do we need an option to specify this behavior? If the LRP is a
> > > > > > > > distributed gateway port and there is no localnet port on the
> > > > > > > > connected LS, can't we just do the centralized routing?
> > > > > > >
> > > > > > > That was my initial approach.  But many test cases fail [1] with this approach.
> > > > > > > It was hard to fix all these test cases and I was not sure if I'd
> > > > > > > break any existing
> > > > > > > scenarios.
> > > > > > >
> > > > > > > [1] https://github.com/numansiddique/ovn/actions/runs/10275831079/job/28435316245
> > > > > > > testsuite: 75 76 161 162 163 164 165 166 167 168 169 170 183 184 185
> > > > > > > 186 187 188 271 272 291 292 failed
> > > > > > >
> > > > > >
> > > > > > I am a little concerned about this. The original behavior is like a
> > > > > > bug, because a LSP on the external overlay network is supposed to be
> > > > > > able to communicate with the LSPs in the internal overlay networks. So
> > > > > > I understand that this patch is trying to fix that. However, the fix
> > > > > > would break some scenarios and it is not very clear why they break
> > > > > > what scenarios would break. Because of this you added a new option to
> > > > > > workaround the problem, because the test cases that didn't enable this
> > > > > > option would still pass for sure. However, it also means that when
> > > > > > this option is enabled, we are not very sure what other scenarios
> > > > > > would break. Would it be better to examine the failed cases to
> > > > > > understand what is really broken?
> > > > >
> > > > > From my usage and tests all of these should be fixed by
> > > > > https://mail.openvswitch.org/pipermail/ovs-dev/2024-August/416264.html
> > > > >
> > > > > Then i guess the option can be removed as well.
> > > > >
> > > >
> > > > Thanks Felix. Hope this solves the problem.
> > > >
> > > > Numan, I don't have any other comment to the code except that the word
> > > > "chassisresident" in code (and also in the commit message) may be
> > > > confusing. We have been using the term chassisredirect or
> > > > chassis-redirect which is what "CR" stands for. I see only one
> > > > occurrence of chassisresident which is in the comment of is_cr_port()
> > > > and I believe that was a typo. So would be better to unify the term
> > > > and stick to chassisredirect in the var namings and comments.
> > > >
> > > > I will wait for v3 and like to do some tests myself.
> > >
> > > Looks like @Mark Michelson  didn't follow the discussion here and
> > > merged the patch to main.
> > >
> > oops :D
> >
> > > I'll submit another patch to fix the crash reported by @Felix Hüttner
> > > and also address the "chassisresident" comment
> > > from @Han Zhou .
> > >
> > > @Han Zhou   - Regarding the option,  I'm a little concerned about
> > > enabling this behavior by default.
> > > If we enable this patch's behavior by default, then we will create a
> > > chassis resident port for
> > > the  gateway router port's peer.  By looking into the number of test
> > > failures,  I was a bit concerned that
> > > this may break any existing deployments and hence added the option.
> > >
> >
> > Ok, I think of it as a bug fix but it sounds like an experimental
> > feature. From the code it looks ok to me to create a chassis-redirect
> > port for DGP's peer. I understand this may be risky because the change
> > seems big. Introducing an option reduces that risk, but we need to
> > keep in mind that for the use cases that require
> > centralize_routing=true, the risk still exists, unless we figure out
> > all the failed cases and we are sure those failed scenarios are not
> > required for the centralize_routing=true use cases. At that time I'd
> > rather remove the option instead of just changing the default to true,
> > to avoid the burden of maintaining such an option if it SHOULD be true
> > from a design point of view.
>
> Sounds good to me.
>
> How about I submit another patch to mark it as experimental once this patch
> is merged - https://patchwork.ozlabs.org/project/ovn/patch/20240802155449.3528777-1-mmichels@redhat.com/
>  ?
> And later remove the option altogether after addressing all the failed
> scenarios ?
>

Sounds good to me.

Thanks,
Han

> Numan
>
> >
> > Thanks,
> > Han
> >
> > > We can definitely debug the test failures and make this option true by
> > > default, post 24.09.
> > > I'll look into that post 24.09 release (also a little short of cycles
> > > as I want to focus on some reviews).
> > >
> > > Thanks
> > > Numan
> > >
> > >
> > > >
> > > > Thanks,
> > > > Han
> > > >
> > > > > Thanks
> > > > > Felix
> > > > >
> > > > _______________________________________________
> > > > dev mailing list
> > > > dev@openvswitch.org
> > > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
> > _______________________________________________
> > dev mailing list
> > dev@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
diff mbox series

Patch

diff --git a/NEWS b/NEWS
index 87e326f21e..8440a74677 100644
--- a/NEWS
+++ b/NEWS
@@ -47,6 +47,9 @@  Post v24.03.0
   - Add support for CT zone limit that can be specified per LR
     (options:ct-zone-limit), LS (other_config:ct-zone-limit) or LSP
     (options:ct-zone-limit).
+  - A new LRP option 'centralize_routing' has been added to a
+    distributed gateway port to centralize routing if the logical
+    switch of its peer doesn't have a localnet port.
 
 OVN v24.03.0 - 01 Mar 2024
 --------------------------
diff --git a/controller/physical.c b/controller/physical.c
index 3c0200c383..9e04ad5f22 100644
--- a/controller/physical.c
+++ b/controller/physical.c
@@ -1610,6 +1610,10 @@  consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name,
                                                     ct_zones);
             put_zones_ofpacts(&zone_ids, ofpacts_p);
 
+            /* Clear the MFF_INPORT.  Its possible that the same packet may
+             * go out from the same tunnel inport. */
+            put_load(ofp_to_u16(OFPP_NONE), MFF_IN_PORT, 0, 16, ofpacts_p);
+
             /* Resubmit to table 41. */
             put_resubmit(OFTABLE_CHECK_LOOPBACK, ofpacts_p);
         }
diff --git a/northd/northd.c b/northd/northd.c
index 5c2fd74ff1..4f59d4f1a3 100644
--- a/northd/northd.c
+++ b/northd/northd.c
@@ -2107,6 +2107,55 @@  parse_lsp_addrs(struct ovn_port *op)
     }
 }
 
+static struct ovn_port *
+create_cr_port(struct ovn_port *op, struct hmap *ports,
+               struct ovs_list *both_dbs, struct ovs_list *nb_only)
+{
+    char *redirect_name = ovn_chassis_redirect_name(
+        op->nbsp ? op->nbsp->name : op->nbrp->name);
+
+    struct ovn_port *crp = ovn_port_find(ports, redirect_name);
+    if (crp && crp->sb && crp->sb->datapath == op->od->sb) {
+        ovn_port_set_nb(crp, NULL, op->nbrp);
+        ovs_list_remove(&crp->list);
+        ovs_list_push_back(both_dbs, &crp->list);
+    } else {
+        crp = ovn_port_create(ports, redirect_name,
+                              op->nbsp, op->nbrp, NULL);
+        ovs_list_push_back(nb_only, &crp->list);
+    }
+
+    crp->primary_port = op;
+    op->cr_port = crp;
+    crp->od = op->od;
+    free(redirect_name);
+
+    return crp;
+}
+
+/* Returns true if chassis resident port needs to be created for
+ * op's peer logical switch.  False otherwise.
+ *
+ * Chassis resident port needs to be created if the following
+ * conditionsd are met:
+ *   - op is a distributed gateway port
+ *   - op has the option 'centralize_routing' set to true
+ *   - op is the only distributed gateway port attached to its
+ *     router
+ *   - op's peer logical switch has no localnet ports.
+ */
+static bool
+peer_needs_cr_port_creation(struct ovn_port *op)
+{
+    if ((op->nbrp->n_gateway_chassis || op->nbrp->ha_chassis_group)
+        && op->od->n_l3dgw_ports == 1 && op->peer && op->peer->nbsp
+        && !op->peer->od->n_localnet_ports) {
+        return smap_get_bool(&op->nbrp->options, "centralize_routing", false);
+    }
+
+    return false;
+}
+
 static void
 join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
                    struct hmap *ls_datapaths, struct hmap *lr_datapaths,
@@ -2214,9 +2263,10 @@  join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
             tag_alloc_add_existing_tags(tag_alloc_table, nbsp);
         }
     }
+
+    struct hmapx dgps = HMAPX_INITIALIZER(&dgps);
     HMAP_FOR_EACH (od, key_node, lr_datapaths) {
         ovs_assert(od->nbr);
-        size_t n_allocated_l3dgw_ports = 0;
         for (size_t i = 0; i < od->nbr->n_ports; i++) {
             const struct nbrec_logical_router_port *nbrp
                 = od->nbr->ports[i];
@@ -2280,10 +2330,7 @@  join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
                     redirect_type && !strcasecmp(redirect_type, "bridged");
             }
 
-            if (op->nbrp->ha_chassis_group ||
-                op->nbrp->n_gateway_chassis) {
-                /* Additional "derived" ovn_port crp represents the
-                 * instance of op on the gateway chassis. */
+            if (op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis) {
                 const char *gw_chassis = smap_get(&op->od->nbr->options,
                                                "chassis");
                 if (gw_chassis) {
@@ -2292,34 +2339,9 @@  join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
                     VLOG_WARN_RL(&rl, "Bad configuration: distributed "
                                  "gateway port configured on port %s "
                                  "on L3 gateway router", nbrp->name);
-                    continue;
-                }
-
-                char *redirect_name =
-                    ovn_chassis_redirect_name(nbrp->name);
-                struct ovn_port *crp = ovn_port_find(ports, redirect_name);
-                if (crp && crp->sb && crp->sb->datapath == od->sb) {
-                    ovn_port_set_nb(crp, NULL, nbrp);
-                    ovs_list_remove(&crp->list);
-                    ovs_list_push_back(both, &crp->list);
                 } else {
-                    crp = ovn_port_create(ports, redirect_name,
-                                          NULL, nbrp, NULL);
-                    ovs_list_push_back(nb_only, &crp->list);
-                }
-                crp->primary_port = op;
-                op->cr_port = crp;
-                crp->od = od;
-                free(redirect_name);
-
-                /* Add to l3dgw_ports in od, for later use during flow
-                 * creation. */
-                if (od->n_l3dgw_ports == n_allocated_l3dgw_ports) {
-                    od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
-                                                 &n_allocated_l3dgw_ports,
-                                                 sizeof *od->l3dgw_ports);
+                    hmapx_add(&dgps, op);
                 }
-                od->l3dgw_ports[od->n_l3dgw_ports++] = op;
            }
         }
     }
@@ -2376,12 +2398,6 @@  join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
                         arp_proxy, op->nbsp->name);
                 }
             }
-
-            /* Only used for the router type LSP whose peer is l3dgw_port */
-            if (op->peer && is_l3dgw_port(op->peer)) {
-                op->enable_router_port_acl = smap_get_bool(
-                    &op->nbsp->options, "enable_router_port_acl", false);
-            }
         } else if (op->nbrp && op->nbrp->peer && !is_cr_port(op)) {
             struct ovn_port *peer = ovn_port_find(ports, op->nbrp->peer);
             if (peer) {
@@ -2402,6 +2418,57 @@  join_logical_ports(const struct sbrec_port_binding_table *sbrec_pb_table,
         }
     }
 
+    struct hmapx_node *hmapx_node;
+    HMAPX_FOR_EACH (hmapx_node, &dgps) {
+        op = hmapx_node->data;
+        od = op->od;
+        ovs_assert(op->nbrp);
+        ovs_assert(op->nbrp->ha_chassis_group || op->nbrp->n_gateway_chassis);
+
+        /* Additional "derived" ovn_port crp represents the instance of op on
+         * the gateway chassis. */
+        struct ovn_port *crp = create_cr_port(op, ports, both, nb_only);
+        ovs_assert(crp);
+
+        /* Add to l3dgw_ports in od, for later use during flow creation. */
+        if (od->n_l3dgw_ports == od->n_allocated_l3dgw_ports) {
+            od->l3dgw_ports = x2nrealloc(od->l3dgw_ports,
+                                        &od->n_allocated_l3dgw_ports,
+                                        sizeof *od->l3dgw_ports);
+        }
+        od->l3dgw_ports[od->n_l3dgw_ports++] = op;
+
+        if (op->peer && op->peer->nbsp) {
+            /* Only used for the router type LSP whose peer is l3dgw_port */
+            op->peer->enable_router_port_acl = smap_get_bool(
+                    &op->peer->nbsp->options, "enable_router_port_acl", false);
+        }
+    }
+
+
+    /* Create chassisresident port for the distributed gateway port's (DGP)
+     * peer if
+     *  - DGP's router has only one DGP and
+     *  - Its peer is a logical switch port and
+     *  - It's peer's logical switch has no localnet ports and
+     *  - option 'centralize_routing' is set to true for the DGP.
+     *
+     * This is required to support
+     *   - NAT via geneve (for the overlay provider networks) and
+     *   - to centralize routing on the gateway chassis for the traffic
+     *     destined to the DGP's networks.
+     *
+     * Future enhancement: Support 'centralizerouting' for all the DGP's
+     * of a logical router.
+     * */
+    HMAPX_FOR_EACH (hmapx_node, &dgps) {
+        op = hmapx_node->data;
+        if (peer_needs_cr_port_creation(op)) {
+            create_cr_port(op->peer, ports, both, nb_only);
+        }
+    }
+    hmapx_destroy(&dgps);
+
     /* Wait until all ports have been connected to add to IPAM since
      * it relies on proper peers to be set
      */
@@ -3184,16 +3251,28 @@  ovn_port_update_sbrec(struct ovsdb_idl_txn *ovnsb_txn,
              * type "l3gateway". */
             if (chassis) {
                 sbrec_port_binding_set_type(op->sb, "l3gateway");
+            } else if (is_cr_port(op)) {
+                sbrec_port_binding_set_type(op->sb, "chassisredirect");
+                ovs_assert(op->primary_port->peer);
+                ovs_assert(op->primary_port->peer->cr_port);
+                ovs_assert(op->primary_port->peer->cr_port->sb);
+                sbrec_port_binding_set_ha_chassis_group(
+                    op->sb,
+                    op->primary_port->peer->cr_port->sb->ha_chassis_group);
+
             } else {
                 sbrec_port_binding_set_type(op->sb, "patch");
             }
 
             const char *router_port = smap_get(&op->nbsp->options,
                                                "router-port");
-            if (router_port || chassis) {
+            if (router_port || chassis || is_cr_port(op)) {
                 struct smap new;
                 smap_init(&new);
-                if (router_port) {
+
+                if (is_cr_port(op)) {
+                    smap_add(&new, "distributed-port", op->nbsp->name);
+                } else if (router_port) {
                     smap_add(&new, "peer", router_port);
                 }
                 if (chassis) {
@@ -8155,9 +8234,27 @@  build_lswitch_rport_arp_req_flow(
     struct lflow_ref *lflow_ref)
 {
     struct ds match   = DS_EMPTY_INITIALIZER;
+    struct ds m       = DS_EMPTY_INITIALIZER;
     struct ds actions = DS_EMPTY_INITIALIZER;
 
-    arp_nd_ns_match(ips, addr_family, &match);
+    arp_nd_ns_match(ips, addr_family, &m);
+    ds_clone(&match, &m);
+
+    bool has_cr_port = patch_op->cr_port;
+
+    /* If the patch_op has a chassis resident port, it means
+     *    - its peer is a distributed gateway port (DGP) and
+     *    - routing is centralized for the DGP's networks on
+     *      the configured gateway chassis.
+     *
+     * If that's the case, make sure that the packets destined to
+     * the DGP's MAC are sent to the chassis where the DGP resides.
+     * */
+
+    if (has_cr_port) {
+        ds_put_format(&match, " && is_chassis_resident(%s)",
+                      patch_op->cr_port->json_key);
+    }
 
     /* Send a the packet to the router pipeline.  If the switch has non-router
      * ports then flood it there as well.
@@ -8179,6 +8276,31 @@  build_lswitch_rport_arp_req_flow(
                                 lflow_ref);
     }
 
+    if (has_cr_port) {
+        ds_clear(&match);
+        ds_put_format(&match, "%s && !is_chassis_resident(%s)", ds_cstr(&m),
+                      patch_op->cr_port->json_key);
+        ds_clear(&actions);
+        if (od->n_router_ports != od->nbs->n_ports) {
+            ds_put_format(&actions, "clone {outport = %s; output; }; "
+                                    "outport = \""MC_FLOOD_L2"\"; output;",
+                          patch_op->cr_port->json_key);
+            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
+                                    priority, ds_cstr(&match),
+                                    ds_cstr(&actions), stage_hint,
+                                    lflow_ref);
+        } else {
+            ds_put_format(&actions, "outport = %s; output;",
+                          patch_op->cr_port->json_key);
+            ovn_lflow_add_with_hint(lflows, od, S_SWITCH_IN_L2_LKUP,
+                                    priority, ds_cstr(&match),
+                                    ds_cstr(&actions),
+                                    stage_hint,
+                                    lflow_ref);
+        }
+    }
+
+    ds_destroy(&m);
     ds_destroy(&match);
     ds_destroy(&actions);
 }
@@ -9548,7 +9670,11 @@  build_lswitch_ip_unicast_lookup(struct ovn_port *op,
                                 struct ds *actions, struct ds *match)
 {
     ovs_assert(op->nbsp);
-    if (lsp_is_external(op->nbsp)) {
+
+    /* Note: A switch port can also have a chassis resident derived port.
+     * Check if 'op' is a chassis resident dervied port. If so, skip
+     * adding unicast lookup flows for this port. */
+    if (lsp_is_external(op->nbsp) || is_cr_port(op)) {
         return;
     }
 
@@ -9566,8 +9692,6 @@  build_lswitch_ip_unicast_lookup(struct ovn_port *op,
                            "outport = \""MC_UNKNOWN "\"; output;"
                          : "outport = %s; output;")
                          : debug_drop_action();
-    ds_clear(actions);
-    ds_put_format(actions, action, op->json_key);
 
     if (lsp_is_router(op->nbsp) && op->peer && op->peer->nbrp) {
         /* For ports connected to logical routers add flows to bypass the
@@ -9614,14 +9738,43 @@  build_lswitch_ip_unicast_lookup(struct ovn_port *op,
             if (add_chassis_resident_check) {
                 ds_put_format(match, " && is_chassis_resident(%s)", json_key);
             }
+        } else if (op->cr_port) {
+            /* If the op has a chassis resident port, it means
+             *   - its peer is a distributed gateway port (DGP) and
+             *   - routing is centralized for the DGP's networks on
+             *     the configured gateway chassis.
+             *
+             * If that's the case, make sure that the packets destined to
+             * the DGP's MAC are sent to the chassis where the DGP resides.
+             * */
+            ds_clear(actions);
+            ds_put_format(actions, action, op->cr_port->json_key);
+
+            struct ds m = DS_EMPTY_INITIALIZER;
+            ds_put_format(&m, "eth.dst == %s && !is_chassis_resident(%s)",
+                          op->peer->lrp_networks.ea_s,
+                          op->cr_port->json_key);
+
+            ovn_lflow_add_with_hint(lflows, op->od,
+                                    S_SWITCH_IN_L2_LKUP, 50,
+                                    ds_cstr(&m), ds_cstr(actions),
+                                    &op->nbsp->header_,
+                                    op->lflow_ref);
+            ds_destroy(&m);
+            ds_put_format(match, " && is_chassis_resident(%s)",
+                          op->cr_port->json_key);
         }
 
+        ds_clear(actions);
+        ds_put_format(actions, action, op->json_key);
         ovn_lflow_add_with_hint(lflows, op->od,
                                 S_SWITCH_IN_L2_LKUP, 50,
                                 ds_cstr(match), ds_cstr(actions),
                                 &op->nbsp->header_,
                                 op->lflow_ref);
     } else {
+        ds_clear(actions);
+        ds_put_format(actions, action, op->json_key);
         for (size_t i = 0; i < op->n_lsp_addrs; i++) {
             ds_clear(match);
             ds_put_format(match, "eth.dst == %s", op->lsp_addrs[i].ea_s);
@@ -11725,6 +11878,14 @@  build_lrouter_port_nat_arp_nd_flow(struct ovn_port *op,
         return;
     }
 
+    if (op->peer && op->peer->cr_port) {
+        /* We don't add the below flows if the router port's peer has
+         * a chassisresident port.  That's because routing is centralized on
+         * the gateway chassis for the router port networks/subnets.
+         */
+        return;
+    }
+
     /* Mac address to use when replying to ARP/NS. */
     const char *mac_s = REG_INPORT_ETH_ADDR;
     struct eth_addr mac;
@@ -15109,6 +15270,16 @@  lrouter_check_nat_entry(const struct ovn_datapath *od,
     /* For distributed router NAT, determine whether this NAT rule
      * satisfies the conditions for distributed NAT processing. */
     *distributed = false;
+
+    /* NAT cannnot be distributed if the DGP's peer
+     * has a chassisresident port (as the routing is centralized
+     * on the gateway chassis for the DGP's networks/subnets.)
+     */
+    struct ovn_port *l3dgw_port = *nat_l3dgw_port;
+    if (l3dgw_port && l3dgw_port->peer && l3dgw_port->peer->cr_port) {
+        return 0;
+    }
+
     if (od->n_l3dgw_ports && !strcmp(nat->type, "dnat_and_snat") &&
         nat->logical_port && nat->external_mac) {
         if (eth_addr_from_string(nat->external_mac, mac)) {
diff --git a/northd/northd.h b/northd/northd.h
index d4a8d75abc..d7c9655916 100644
--- a/northd/northd.h
+++ b/northd/northd.h
@@ -325,6 +325,7 @@  struct ovn_datapath {
      * will be NULL. */
     struct ovn_port **l3dgw_ports;
     size_t n_l3dgw_ports;
+    size_t n_allocated_l3dgw_ports;
 
     /* router datapath has a logical port with redirect-type set to bridged. */
     bool redirect_bridged;
diff --git a/ovn-nb.xml b/ovn-nb.xml
index a4362a4ef1..217d1cd1fe 100644
--- a/ovn-nb.xml
+++ b/ovn-nb.xml
@@ -3499,6 +3499,40 @@  or
           <ref column="options" key="gateway_mtu"/> option.
         </p>
       </column>
+
+      <column name="options" key="centralize_routing"
+              type='{"type": "boolean"}'>
+        <p>
+          This option is applicable only if the router port is a
+          distributed gateway port i.e if the <ref table="Logical_Router_Port"
+          column="ha_chassis_group"/> column or
+          <ref table="Logical_Router_Port" column="gateway_chassis"/>
+          is set.
+        </p>
+
+        <p>
+          If set to <code>true</code>, routing for the router port's
+          networks (set in the column <ref table="Logical_Router_Port"
+          column="networks"/>) is centralized on the gateway chassis
+          which claims this distributed gateway port.
+        </p>
+
+        <p>
+          Additionally for this option to take effect, below conditions
+          must be met:
+        </p>
+
+        <ul>
+          <li>
+            The Logical router has only one distributed gateway port.
+          </li>
+
+          <li>
+            The router port's peer logical switch has no localnet ports.
+          </li>
+
+        </ul>
+      </column>
     </group>
 
     <group title="Attachment">
diff --git a/tests/multinode-macros.at b/tests/multinode-macros.at
index 786e564860..757917626c 100644
--- a/tests/multinode-macros.at
+++ b/tests/multinode-macros.at
@@ -92,7 +92,7 @@  m_count_rows() {
 m_check_row_count() {
     local db=$(parse_db $1) table=$(parse_table $1); shift
     local count=$1; shift
-    local found=$(m_count_rows $c $db:$table "$@")
+    local found=$(m_count_rows $db:$table "$@")
     echo
     echo "Checking for $count rows in $db $table${1+ with $*}... found $found"
     if test "$count" != "$found"; then
diff --git a/tests/multinode.at b/tests/multinode.at
index a7231130ac..a725414165 100644
--- a/tests/multinode.at
+++ b/tests/multinode.at
@@ -1033,6 +1033,183 @@  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [sh -c 'dd bs=512 count=2 if=/dev/uran
 done
 M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ip route get 10.0.0.1 dev sw0p1 | grep -q 'mtu 942'])
 
+# Reset back to geneve tunnels
+for c in ovn-chassis-1 ovn-chassis-2 ovn-gw-1
+do
+    m_as $c ovs-vsctl set open . external-ids:ovn-encap-type=geneve
+done
+
+AT_CLEANUP
+
+AT_SETUP([ovn multinode NAT on a provider network with no localnet ports])
+
+# Check that ovn-fake-multinode setup is up and running
+check_fake_multinode_setup
+
+# Delete the multinode NB and OVS resources before starting the test.
+cleanup_multinode_resources
+
+check multinode_nbctl ls-add sw0
+check multinode_nbctl lsp-add sw0 sw0-port1
+check multinode_nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03 10.0.0.3 1000::3"
+check multinode_nbctl lsp-add sw0 sw0-port2
+check multinode_nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:04 10.0.0.4 1000::4"
+
+m_as ovn-chassis-1 /data/create_fake_vm.sh sw0-port1 sw0p1 50:54:00:00:00:03 10.0.0.3 24 10.0.0.1 1000::3/64 1000::a
+m_as ovn-chassis-2 /data/create_fake_vm.sh sw0-port2 sw0p2 50:54:00:00:00:04 10.0.0.4 24 10.0.0.1 1000::4/64 1000::a
+
+m_wait_for_ports_up
+
+M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.4 | FORMAT_PING], \
+[0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+
+# Create the second logical switch with one port
+check multinode_nbctl ls-add sw1
+check multinode_nbctl lsp-add sw1 sw1-port1
+check multinode_nbctl lsp-set-addresses sw1-port1 "40:54:00:00:00:03 20.0.0.3 2000::3"
+
+# Create a logical router and attach both logical switches
+check multinode_nbctl lr-add lr0
+check multinode_nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 1000::a/64
+check multinode_nbctl lsp-add sw0 sw0-lr0
+check multinode_nbctl lsp-set-type sw0-lr0 router
+check multinode_nbctl lsp-set-addresses sw0-lr0 router
+check multinode_nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
+
+check multinode_nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:02 20.0.0.1/24 2000::a/64
+check multinode_nbctl lsp-add sw1 sw1-lr0
+check multinode_nbctl lsp-set-type sw1-lr0 router
+check multinode_nbctl lsp-set-addresses sw1-lr0 router
+check multinode_nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
+
+m_as ovn-chassis-2 /data/create_fake_vm.sh sw1-port1 sw1p1 40:54:00:00:00:03 20.0.0.3 24 20.0.0.1 2000::3/64 2000::a
+
+# create exteranl connection for N/S traffic
+check multinode_nbctl ls-add public
+check multinode_nbctl lsp-add public ln-public
+check multinode_nbctl lsp-set-type ln-public localnet
+check multinode_nbctl lsp-set-addresses ln-public unknown
+check multinode_nbctl lsp-set-options ln-public network_name=public
+
+check multinode_nbctl lrp-add lr0 lr0-public 00:11:22:00:ff:01 172.20.0.100/24
+check multinode_nbctl lsp-add public public-lr0
+check multinode_nbctl lsp-set-type public-lr0 router
+check multinode_nbctl lsp-set-addresses public-lr0 router
+check multinode_nbctl lsp-set-options public-lr0 router-port=lr0-public
+check multinode_nbctl lrp-set-gateway-chassis lr0-public ovn-gw-1 10
+
+check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
+check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.20.0.120 20.0.0.3
+check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 10.0.0.0/24
+check multinode_nbctl lr-nat-add lr0 snat 172.20.0.100 20.0.0.0/24
+
+# Create a logical port pub-p1 and bind it in ovn-chassis-1
+check multinode_nbctl lsp-add public public-port1
+check multinode_nbctl lsp-set-addresses public-port1 "60:54:00:00:00:03 172.168.0.50"
+
+m_as ovn-chassis-1 /data/create_fake_vm.sh public-port1 pubp1 60:54:00:00:00:03 172.20.0.50 24 172.20.0.100
+
+check multinode_nbctl --wait=hv sync
+
+# First do basic ping tests before deleting the localnet port - ln-public.
+# Once the localnet port is deleted from public ls, routing for 172.20.0.0/24
+# is centralized on ovn-gw-1.
+
+# This function checks the North-South traffic.
+run_ns_traffic() {
+  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.110], [ignore], [ignore])
+  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [arp -d 172.20.0.120], [ignore], [ignore])
+
+  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
+[0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+
+  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
+[0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+
+  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
+[0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+
+  M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
+[0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+
+  M_NS_CHECK_EXEC([ovn-chassis-2], [sw1p1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.50 | FORMAT_PING], \
+[0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+
+  # Now ping from pubp1 to 172.20.0.100, 172.20.0.110, 172.20.0.120, 10.0.0.3 and 20.0.0.3
+  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.100 | FORMAT_PING], \
+[0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+
+  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.110 | FORMAT_PING], \
+[0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+
+  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 172.20.0.120 | FORMAT_PING], \
+[0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+
+  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 10.0.0.3 | FORMAT_PING], \
+[0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+
+  M_NS_CHECK_EXEC([ovn-chassis-1], [pubp1], [ping -q -c 3 -i 0.3 -w 2 20.0.0.3 | FORMAT_PING], \
+[0], [dnl
+3 packets transmitted, 3 received, 0% packet loss, time 0ms
+])
+}
+
+# Test out the N-S traffic.
+run_ns_traffic
+
+# Delete the localnet port by changing the type of ln-public to VIF port.
+check multinode_nbctl --wait=hv lsp-set-type ln-public ""
+
+# cr-port should not be created for public-lr0 since the option
+# centralize_routing=true is not yet set for lr0-public.
+m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
+
+# Set the option - centralize_routing now.
+check multinode_nbctl --wait=hv set logical_router_port lr0-public options:centralize_routing=true
+
+m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
+m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
+
+# Test out the N-S traffic.
+run_ns_traffic
+
+# Re-add the localnet port
+check multinode_nbctl --wait=hv lsp-set-type ln-public localnet
+
+m_check_row_count Port_Binding 0 logical_port=cr-public-lr0
+
+# Test out the N-S traffic.
+run_ns_traffic
+
+# Delete the ln-public port this time.
+check multinode_nbctl --wait=hv lsp-del ln-public
+
+m_check_row_count Port_Binding 1 logical_port=cr-public-lr0
+m_check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
+
+# Test out the N-S traffic.
+run_ns_traffic
+
 AT_CLEANUP
 
 AT_SETUP([ovn provider network - always_tunnel])
diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at
index 57f89a7746..649c20f285 100644
--- a/tests/ovn-northd.at
+++ b/tests/ovn-northd.at
@@ -2187,7 +2187,7 @@  match=(inport == "lrp-public" && arp.op == 1 && arp.tpa == 43.43.43.4 && is_chas
 action=(eth.dst = eth.src; eth.src = 00:00:00:00:00:02; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 00:00:00:00:00:02; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
 ])
 
-# xreg0[0..47] isn't used anywhere else.
+# xreg0[[0..47]] isn't used anywhere else.
 AT_CHECK([ovn-sbctl lflow-list | grep "xreg0\[[0..47\]]" | grep -vE 'lr_in_admission|lr_in_ip_input'], [1], [])
 
 AT_CLEANUP
@@ -5503,13 +5503,14 @@  AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | grep "192.168.4.100" | grep "_MC_flo
 
 AS_BOX([Configuring ro1-ls1 router port as a gateway router port])
 
-ovn-nbctl --wait=sb lrp-set-gateway-chassis ro1-ls1 chassis-1 30
+check ovn-nbctl  lrp-set-gateway-chassis ro1-ls1 chassis-1 30
+check ovn-nbctl --wait=sb lsp-add ls1 ln-ls1 -- lsp-set-type ln-ls1 localnet
 
 ovn-sbctl lflow-list ls1 > ls1_lflows
 AT_CHECK([grep "ls_in_l2_lkup" ls1_lflows | ovn_strip_lflows], [0], [dnl
   table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
   table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
-  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01), action=(outport = "ls1-ro1"; output;)
+  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:01 && is_chassis_resident("cr-ro1-ls1")), action=(outport = "ls1-ro1"; output;)
   table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:01:02), action=(outport = "vm1"; output;)
   table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
   table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:01:01} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
@@ -12475,3 +12476,512 @@  AT_CHECK([ovn-sbctl dump-flows lr | grep lr_in_dnat | ovn_strip_lflows], [0], [d
 
 AT_CLEANUP
 ])
+
+OVN_FOR_EACH_NORTHD_NO_HV([
+AT_SETUP([NAT on a provider network with no localnet ports])
+AT_KEYWORDS([NAT])
+ovn_start
+
+check ovn-nbctl -- ls-add sw0 -- ls-add sw1
+check ovn-nbctl lsp-add sw0 sw0-port1
+check ovn-nbctl lr-add lr0
+check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24
+check ovn-nbctl lsp-add sw0 sw0-lr0
+check ovn-nbctl lsp-set-type sw0-lr0 router
+check ovn-nbctl lsp-set-addresses sw0-lr0 00:00:00:00:ff:01
+check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
+
+check ovn-nbctl lrp-add lr0 lr0-sw1 00:00:00:00:ff:03 20.0.0.1/24
+check ovn-nbctl lsp-add sw1 sw1-lr0
+check ovn-nbctl lsp-set-type sw1-lr0 router
+check ovn-nbctl lsp-set-addresses sw1-lr0 router
+check ovn-nbctl lsp-set-options sw1-lr0 router-port=lr0-sw1
+
+check ovn-sbctl chassis-add gw1 geneve 127.0.0.1
+check ovn-nbctl ls-add public
+check ovn-nbctl lsp-add public pub-p1
+
+# localnet port
+check ovn-nbctl lsp-add public ln-public
+check ovn-nbctl lsp-set-type ln-public localnet
+check ovn-nbctl lsp-set-addresses ln-public unknown
+check ovn-nbctl lsp-set-options ln-public network_name=public
+
+check ovn-nbctl lrp-add lr0 lr0-public 00:00:00:00:ff:02 172.168.0.10/24
+check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1
+
+check ovn-nbctl lsp-add public public-lr0
+check ovn-nbctl lsp-set-type public-lr0 router
+check ovn-nbctl lsp-set-addresses public-lr0 router
+check ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public
+
+check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.3 sw0-port1 30:54:00:00:00:03
+check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.120 20.0.0.3
+check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 10.0.0.0/24
+check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 20.0.0.0/24
+
+check ovn-nbctl --wait=sb sync
+
+check_flows_no_cr_port_for_public_lr0() {
+  # check that there is no port binding cr-public-lr0
+  check_row_count Port_Binding 0 logical_port=cr-public-lr0
+
+  ovn-sbctl dump-flows lr0 > lr0flows
+  ovn-sbctl dump-flows public > publicflows
+
+AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
+  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
+  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
+  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
+  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
+  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
+  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
+  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
+])
+
+AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
+  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
+  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
+  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
+  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
+  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
+  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
+  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
+  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
+  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
+  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
+  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
+  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
+  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
+  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
+  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+])
+
+AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
+])
+
+AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
+])
+
+AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
+  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
+  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
+])
+
+AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
+  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
+  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
+  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
+])
+
+AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
+  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
+  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
+])
+
+AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
+  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
+  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
+])
+
+AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
+])
+
+AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
+  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
+  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
+  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
+  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
+  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
+])
+
+AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+])
+
+AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
+  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
+  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
+  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-lr0-public")), action=(outport = "public-lr0"; output;)
+  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
+  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
+  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+])
+
+AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
+  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
+  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
+  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
+  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
+  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
+  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
+])
+
+AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
+  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
+  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+])
+}
+
+check_flows_cr_port_for_public_lr0() {
+  # check that there is port binding cr-public-lr0
+  check_row_count Port_Binding 1 logical_port=cr-public-lr0
+  check_column chassisredirect Port_Binding type logical_port=cr-public-lr0
+
+  ovn-sbctl dump-flows lr0 > lr0flows
+  ovn-sbctl dump-flows public > publicflows
+
+AT_CHECK([grep "lr_in_admission" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_admission    ), priority=0    , match=(1), action=(drop;)
+  table=??(lr_in_admission    ), priority=100  , match=(vlan.present || eth.src[[40]]), action=(drop;)
+  table=??(lr_in_admission    ), priority=110  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && flags.tunnel_rx == 1), action=(drop;)
+  table=??(lr_in_admission    ), priority=120  , match=(((ip4 && icmp4.type == 3 && icmp4.code == 4) || (ip6 && icmp6.type == 2 && icmp6.code == 0)) && eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-lr0-public") && flags.tunnel_rx == 1), action=(outport <-> inport; inport = "lr0-public"; next;)
+  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:01 && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
+  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 00:00:00:00:ff:03 && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
+  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-public"), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw0"), action=(xreg0[[0..47]] = 00:00:00:00:ff:01; next;)
+  table=??(lr_in_admission    ), priority=50   , match=(eth.mcast && inport == "lr0-sw1"), action=(xreg0[[0..47]] = 00:00:00:00:ff:03; next;)
+])
+
+AT_CHECK([grep "lr_in_ip_input" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_ip_input     ), priority=0    , match=(1), action=(next;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {10.0.0.1, 10.0.0.255} && reg9[[0]] == 0), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {172.168.0.10, 172.168.0.255} && reg9[[0]] == 0), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src == {20.0.0.1, 20.0.0.255} && reg9[[0]] == 0), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip4.src_mcast ||ip4.src == 255.255.255.255 || ip4.src == 127.0.0.0/8 || ip4.dst == 127.0.0.0/8 || ip4.src == 0.0.0.0/8 || ip4.dst == 0.0.0.0/8), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff01 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff02 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
+  table=??(lr_in_ip_input     ), priority=100  , match=(ip6.dst == fe80::200:ff:fe00:ff03 && udp.src == 547 && udp.dst == 546), action=(reg0 = 0; handle_dhcpv6_reply;)
+  table=??(lr_in_ip_input     ), priority=120  , match=(inport == "lr0-public" && ip4.src == 172.168.0.100), action=(next;)
+  table=??(lr_in_ip_input     ), priority=30   , match=(ip.ttl == {0, 1}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-public" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst <-> ip4.src ; ip.ttl = 254; outport = "lr0-public"; flags.loopback = 1; output; };)
+  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw0" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 10.0.0.1 ; ip.ttl = 254; outport = "lr0-sw0"; flags.loopback = 1; output; };)
+  table=??(lr_in_ip_input     ), priority=31   , match=(inport == "lr0-sw1" && ip4 && ip.ttl == {0, 1} && !ip.later_frag), action=(icmp4 {eth.dst <-> eth.src; icmp4.type = 11; /* Time exceeded */ icmp4.code = 0; /* TTL exceeded in transit */ ip4.dst = ip4.src; ip4.src = 20.0.0.1 ; ip.ttl = 254; outport = "lr0-sw1"; flags.loopback = 1; output; };)
+  table=??(lr_in_ip_input     ), priority=32   , match=(ip.ttl == {0, 1} && !ip.later_frag && (ip4.mcast || ip6.mcast)), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=50   , match=(eth.bcast), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {10.0.0.1}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {172.168.0.10}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=60   , match=(ip4.dst == {20.0.0.1}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff01}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff02}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=60   , match=(ip6.dst == {fe80::200:ff:fe00:ff03}), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=82   , match=(ip4.mcast || ip6.mcast), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=83   , match=(ip6.mcast_rsvd), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=84   , match=(nd_rs || nd_ra), action=(next;)
+  table=??(lr_in_ip_input     ), priority=85   , match=(arp || nd), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.100), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.10 && arp.spa == 172.168.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-public" && ip6.dst == {fe80::200:ff:fe00:ff02, ff02::1:ff00:ff02} && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-lr0-public")), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
+  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && arp.op == 1 && arp.tpa == 10.0.0.1 && arp.spa == 10.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw0" && ip6.dst == {fe80::200:ff:fe00:ff01, ff02::1:ff00:ff01} && nd_ns && nd.target == fe80::200:ff:fe00:ff01), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
+  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && arp.op == 1 && arp.tpa == 20.0.0.1 && arp.spa == 20.0.0.0/24), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(inport == "lr0-sw1" && ip6.dst == {fe80::200:ff:fe00:ff03, ff02::1:ff00:ff03} && nd_ns && nd.target == fe80::200:ff:fe00:ff03), action=(nd_na_router { eth.src = xreg0[[0..47]]; ip6.src = nd.target; nd.tll = xreg0[[0..47]]; outport = inport; flags.loopback = 1; output; };)
+  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 10.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
+  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 172.168.0.10 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
+  table=??(lr_in_ip_input     ), priority=90   , match=(ip4.dst == 20.0.0.1 && icmp4.type == 8 && icmp4.code == 0), action=(ip4.dst <-> ip4.src; ip.ttl = 255; icmp4.type = 0; flags.loopback = 1; next; )
+  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff01 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
+  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff02 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
+  table=??(lr_in_ip_input     ), priority=90   , match=(ip6.dst == fe80::200:ff:fe00:ff03 && icmp6.type == 128 && icmp6.code == 0), action=(ip6.dst <-> ip6.src; ip.ttl = 255; icmp6.type = 129; flags.loopback = 1; next; )
+])
+
+AT_CHECK([grep "lr_in_unsnat" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_unsnat       ), priority=0    , match=(1), action=(next;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.100 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
+])
+
+AT_CHECK([grep "lr_in_defrag" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_defrag       ), priority=0    , match=(1), action=(next;)
+])
+
+AT_CHECK([grep "lr_in_dnat" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_dnat         ), priority=0    , match=(1), action=(next;)
+  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
+  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
+])
+
+AT_CHECK([grep "lr_in_arp_resolve" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_arp_resolve  ), priority=0    , match=(1), action=(drop;)
+  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip4), action=(get_arp(outport, reg0); next;)
+  table=??(lr_in_arp_resolve  ), priority=1    , match=(ip6), action=(get_nd(outport, xxreg0); next;)
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.100), action=(eth.dst = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.100), action=(drop;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
+  table=??(lr_in_arp_resolve  ), priority=500  , match=(ip4.mcast || ip6.mcast), action=(next;)
+])
+
+AT_CHECK([grep "lr_in_gw_redirect" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_gw_redirect  ), priority=0    , match=(1), action=(next;)
+  table=??(lr_in_gw_redirect  ), priority=50   , match=(outport == "lr0-public"), action=(outport = "cr-lr0-public"; next;)
+])
+
+AT_CHECK([grep "lr_out_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_out_undnat      ), priority=0    , match=(1), action=(next;)
+  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
+  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
+])
+
+AT_CHECK([grep "lr_out_post_undnat" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_out_post_undnat ), priority=0    , match=(1), action=(next;)
+])
+
+AT_CHECK([grep "lr_out_snat" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_out_snat        ), priority=0    , match=(1), action=(next;)
+  table=??(lr_out_snat        ), priority=120  , match=(nd_ns), action=(next;)
+  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 10.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
+  table=??(lr_out_snat        ), priority=153  , match=(ip && ip4.src == 20.0.0.0/24 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.100);)
+  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
+  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
+])
+
+AT_CHECK([grep "lr_out_egr_loop" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_out_egr_loop    ), priority=0    , match=(1), action=(next;)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.100 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+])
+
+AT_CHECK([grep "ls_in_l2_lkup" publicflows | ovn_strip_lflows], [0], [dnl
+  table=??(ls_in_l2_lkup      ), priority=0    , match=(1), action=(outport = get_fdb(eth.dst); next;)
+  table=??(ls_in_l2_lkup      ), priority=110  , match=(eth.dst == $svc_monitor_mac && (tcp || icmp || icmp6)), action=(handle_svc_check(inport);)
+  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && !is_chassis_resident("cr-public-lr0")), action=(outport = "cr-public-lr0"; output;)
+  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 00:00:00:00:ff:02 && is_chassis_resident("cr-public-lr0")), action=(outport = "public-lr0"; output;)
+  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
+  table=??(ls_in_l2_lkup      ), priority=70   , match=(eth.mcast), action=(outport = "_MC_flood"; output;)
+  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.10 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.100 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && nd_ns && nd.target == fe80::200:ff:fe00:ff02 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+])
+
+AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
+  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(10.0.0.3);)
+  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.110);)
+  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
+  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
+  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
+])
+
+AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
+  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
+  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && !is_chassis_resident("cr-public-lr0")), action=(clone {outport = "cr-public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-public-lr0")), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+])
+}
+
+# Check that the lflows are as expected when public has localnet port.
+check_flows_no_cr_port_for_public_lr0
+
+# Remove the localnet port from public logical switch.
+check ovn-nbctl --wait=sb lsp-set-type ln-public ""
+
+# Check that the lflows are as expected and there is no cr port
+# created for "public-lr0"  when public has no localnet port
+# since public doesn't have the option "overlay_provider_network=true"
+# set.
+check_row_count Port_Binding 0 logical_port=cr-public-lr0
+
+ovn-sbctl dump-flows lr0 > lr0flows
+ovn-sbctl dump-flows public > publicflows
+
+AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
+  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
+  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
+  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
+  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
+  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
+  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
+])
+
+AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
+  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
+  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+])
+
+
+# Set the option "centralize_routing=true" for lr0-public.
+check ovn-nbctl --wait=sb set logical_router_port lr0-public options:centralize_routing=true
+
+# Check that the lflows are as expected and there is cr port created for public-lr0.
+check_flows_cr_port_for_public_lr0
+
+# Set the type of ln-public back to localnet
+check ovn-nbctl --wait=sb lsp-set-type ln-public localnet
+
+# Check that the lflows are as expected when public has localnet port.
+check_flows_no_cr_port_for_public_lr0
+
+# Delete the localnet port
+check ovn-nbctl --wait=sb lsp-del ln-public
+
+# Check that the lflows are as expected when public has no localnet port.
+check_flows_cr_port_for_public_lr0
+
+# Create multiple gateway ports.  chassisresident port should not be
+# created for 'public-lr0' even if there is no localnet port on 'public'
+# logical switch.
+check ovn-nbctl --wait=sb lrp-set-gateway-chassis lr0-sw0 gw1
+# check that there is no port binding cr-public-lr0
+check_row_count Port_Binding 0 logical_port=cr-public-lr0
+
+ovn-sbctl dump-flows lr0 > lr0flows
+ovn-sbctl dump-flows public > publicflows
+
+AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" lr0flows | ovn_strip_lflows], [0], [dnl
+  table=??(lr_in_admission    ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && inport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(xreg0[[0..47]] = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.110), action=(eth.dst = 30:54:00:00:00:03; next;)
+  table=??(lr_in_arp_resolve  ), priority=100  , match=(outport == "lr0-public" && reg0 == 172.168.0.120), action=(eth.dst = 00:00:00:00:ff:02; next;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.110), action=(drop;)
+  table=??(lr_in_arp_resolve  ), priority=150  , match=(inport == "lr0-public" && outport == "lr0-public" && ip4.dst == 172.168.0.120), action=(drop;)
+  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_dnat(10.0.0.3);)
+  table=??(lr_in_dnat         ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat(20.0.0.3);)
+  table=??(lr_in_gw_redirect  ), priority=100  , match=(ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(eth.src = 30:54:00:00:00:03; reg1 = 172.168.0.110; next;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.110), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=90   , match=(arp.op == 1 && arp.tpa == 172.168.0.120), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=91   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120), action=(drop;)
+  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.110 && is_chassis_resident("sw0-port1")), action=(eth.dst = eth.src; eth.src = 30:54:00:00:00:03; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = 30:54:00:00:00:03; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_ip_input     ), priority=92   , match=(inport == "lr0-public" && arp.op == 1 && arp.tpa == 172.168.0.120 && is_chassis_resident("cr-lr0-public")), action=(eth.dst = eth.src; eth.src = xreg0[[0..47]]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[[0..47]]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.110 && inport == "lr0-public"), action=(ct_snat;)
+  table=??(lr_in_unsnat       ), priority=100  , match=(ip && ip4.dst == 172.168.0.120 && inport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_snat;)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.110 && outport == "lr0-public" && is_chassis_resident("sw0-port1")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+  table=??(lr_out_egr_loop    ), priority=100  , match=(ip4.dst == 172.168.0.120 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(clone { ct_clear; inport = outport; outport = ""; eth.dst <-> eth.src; flags = 0; flags.loopback = 1; reg0 = 0; reg1 = 0; reg2 = 0; reg3 = 0; reg4 = 0; reg5 = 0; reg6 = 0; reg7 = 0; reg8 = 0; reg9 = 0; reg9[[0]] = 1; next(pipeline=ingress, table=??); };)
+  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public" && is_chassis_resident("sw0-port1") && (!ct.trk || !ct.rpl)), action=(eth.src = 30:54:00:00:00:03; ct_snat(172.168.0.110);)
+  table=??(lr_out_snat        ), priority=161  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public") && (!ct.trk || !ct.rpl)), action=(ct_snat(172.168.0.120);)
+  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 10.0.0.3 && outport == "lr0-public"), action=(eth.src = 30:54:00:00:00:03; ct_dnat;)
+  table=??(lr_out_undnat      ), priority=100  , match=(ip && ip4.src == 20.0.0.3 && outport == "lr0-public" && is_chassis_resident("cr-lr0-public")), action=(ct_dnat;)
+])
+
+AT_CHECK([grep -e "172.168.0.110" -e "172.168.0.120" -e "10.0.0.3" -e "20.0.0.3" -e "30:54:00:00:00:03"  -e "sw0-port1" publicflows | ovn_strip_lflows], [0], [dnl
+  table=??(ls_in_l2_lkup      ), priority=50   , match=(eth.dst == 30:54:00:00:00:03 && is_chassis_resident("sw0-port1")), action=(outport = "public-lr0"; output;)
+  table=??(ls_in_l2_lkup      ), priority=75   , match=(eth.src == {00:00:00:00:ff:02, 30:54:00:00:00:03} && (arp.op == 1 || rarp.op == 3 || nd_ns)), action=(outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.110), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+  table=??(ls_in_l2_lkup      ), priority=80   , match=(flags[[1]] == 0 && arp.op == 1 && arp.tpa == 172.168.0.120), action=(clone {outport = "public-lr0"; output; }; outport = "_MC_flood_l2"; output;)
+])
+
+AT_CLEANUP
+])
diff --git a/tests/ovn.at b/tests/ovn.at
index 582ec56485..c468673f0c 100644
--- a/tests/ovn.at
+++ b/tests/ovn.at
@@ -21227,10 +21227,10 @@  ovn-nbctl lsp-add sw0 rp-sw0 -- set Logical_Switch_Port rp-sw0 \
     type=router options:router-port=sw0 \
     -- lsp-set-addresses rp-sw0 router
 
-ovn-nbctl lrp-add lr0 sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
-    -- lrp-set-gateway-chassis sw1 hv2
+ovn-nbctl lrp-add lr0 lr0-sw1 00:00:02:01:02:03 172.16.1.1/24 2002:0:0:0:0:0:0:1/64 \
+    -- lrp-set-gateway-chassis lr0-sw1 hv2
 ovn-nbctl lsp-add sw1 rp-sw1 -- set Logical_Switch_Port rp-sw1 \
-    type=router options:router-port=sw1 \
+    type=router options:router-port=lr0-sw1 \
     -- lsp-set-addresses rp-sw1 router
 
 ovn-nbctl lsp-add sw0 sw0-p0 \
@@ -21242,6 +21242,8 @@  ovn-nbctl lsp-add sw0 sw0-p1 \
 ovn-nbctl lsp-add sw1 sw1-p0 \
     -- lsp-set-addresses sw1-p0 unknown
 
+check ovn-nbctl lsp-add sw1 ln-sw1 -- lsp-set-type ln-sw1 localnet
+
 ovn-nbctl lr-nat-add lr0 snat 172.16.1.1 192.168.1.0/24
 ovn-nbctl lr-nat-add lr0 snat 2002::1 2001::/64