[ovs-dev,v1,3/3] controller: Optimize adding 'dps' to the local datapaths.

From: Numan Siddique <numans@ovn.org>

Consider the below logical topology

sw0-p1 -
        |
sw0-p2 -   ->  sw0 -> lr0 ----
...     |                     |
sw0-pn -                      |
                              |
sw1-p1 -                      |
        |                     |
sw1p-2 -   ->  sw1 -> lr1 ----  --- public (provider switch)
...     |                     |
sw1-pn-                       |
                              |
swn-p1 -                      |
        |                     |
swn-p2-    ->  swn -> lrn ----
...     |
swn-pn -

All the routers are connected to the provider switch via
a ditributed gateway port.

If sw0-p1 is resident on the chassis C1, then since there is a path
to all the switches and the routers, ovn-controller will add all
these datapaths to its 'local_datapaths' map.  This in turn results
in processing all the logical flows and installing all the openflows
and in turn wasting the CPU time.  This can be very costly in
a highly scaled deployment.

Previous commit sets a flag "only_dgp_peer_ports" in the SB Datapath
binding for a provider switch (with only dgp peer ports).

In this commit, ovn-controller makes use of this flag and stops
adding other datapaths connected to the public provider switch
to the 'local_datapaths'.

For example, when it claims sw0-p1, it adds sw0, lr0 and public
to the local_datapaths and stops there.  If it later claims
sw1-p1, it will add sw1 and lr1.

This reduces the recompute time and the number of openflow rules
added to ovs-vswitchd significantly.

I tested this patch with a deployment of below logical resources:

No of logical switches - 778
No of logical routers  - 871
No of logical flows    - 85626
No of 'ovn-sbctl dump-flows' - 208631

Without this patch, afte claiming sw0-p1, ovn-controller adds
269098 openflow rules and it takes approx 2500 milli seconds
for a recompute.

With this patch, after claiming sw0-p1, ovn-controller adds
21350 openflow rules and it takes approx 280 milli seconds
for a recompute.

There is approx 90% reduction in the openflow rules and
88% reduction in recompute time when a comoute node has
VIFs from one logical switch.

Signed-off-by: Numan Siddique <numans@ovn.org>
---
 controller/binding.c        | 249 +++++++++--
 controller/binding.h        |   2 +
 controller/local_data.c     |  84 +++-
 controller/local_data.h     |   6 +
 controller/lport.c          |  12 +
 controller/lport.h          |   4 +
 controller/ovn-controller.c |  38 ++
 tests/multinode.at          | 178 ++++++++
 tests/ovn-performance.at    |   6 +-
 tests/ovn.at                | 853 ++++++++++++++++++++++++++++++++++++
 10 files changed, 1394 insertions(+), 38 deletions(-)

On Mon, Feb 10, 2025 at 10:37 AM <numans@ovn.org> wrote:
>
> From: Numan Siddique <numans@ovn.org>
>
> Consider the below logical topology
>
> sw0-p1 -
>         |
> sw0-p2 -   ->  sw0 -> lr0 ----
> ...     |                     |
> sw0-pn -                      |
>                               |
> sw1-p1 -                      |
>         |                     |
> sw1p-2 -   ->  sw1 -> lr1 ----  --- public (provider switch)
> ...     |                     |
> sw1-pn-                       |
>                               |
> swn-p1 -                      |
>         |                     |
> swn-p2-    ->  swn -> lrn ----
> ...     |
> swn-pn -
>
> All the routers are connected to the provider switch via
> a ditributed gateway port.
>
> If sw0-p1 is resident on the chassis C1, then since there is a path
> to all the switches and the routers, ovn-controller will add all
> these datapaths to its 'local_datapaths' map.  This in turn results
> in processing all the logical flows and installing all the openflows
> and in turn wasting the CPU time.  This can be very costly in
> a highly scaled deployment.
>
> Previous commit sets a flag "only_dgp_peer_ports" in the SB Datapath
> binding for a provider switch (with only dgp peer ports).
>
> In this commit, ovn-controller makes use of this flag and stops
> adding other datapaths connected to the public provider switch
> to the 'local_datapaths'.
>
> For example, when it claims sw0-p1, it adds sw0, lr0 and public
> to the local_datapaths and stops there.  If it later claims
> sw1-p1, it will add sw1 and lr1.
>
> This reduces the recompute time and the number of openflow rules
> added to ovs-vswitchd significantly.
>
> I tested this patch with a deployment of below logical resources:
>
> No of logical switches - 778
> No of logical routers  - 871
> No of logical flows    - 85626
> No of 'ovn-sbctl dump-flows' - 208631
>
> Without this patch, afte claiming sw0-p1, ovn-controller adds
> 269098 openflow rules and it takes approx 2500 milli seconds
> for a recompute.
>
> With this patch, after claiming sw0-p1, ovn-controller adds
> 21350 openflow rules and it takes approx 280 milli seconds
> for a recompute.
>
> There is approx 90% reduction in the openflow rules and
> 88% reduction in recompute time when a comoute node has
> VIFs from one logical switch.
>
> Signed-off-by: Numan Siddique <numans@ovn.org>
> ---
>  controller/binding.c        | 249 +++++++++--
>  controller/binding.h        |   2 +
>  controller/local_data.c     |  84 +++-
>  controller/local_data.h     |   6 +
>  controller/lport.c          |  12 +
>  controller/lport.h          |   4 +
>  controller/ovn-controller.c |  38 ++
>  tests/multinode.at          | 178 ++++++++
>  tests/ovn-performance.at    |   6 +-
>  tests/ovn.at                | 853 ++++++++++++++++++++++++++++++++++++
>  10 files changed, 1394 insertions(+), 38 deletions(-)
>
> diff --git a/controller/binding.c b/controller/binding.c
> index ea5bf5a9fd..88c9640c3a 100644
> --- a/controller/binding.c
> +++ b/controller/binding.c
> @@ -825,6 +825,16 @@ static bool binding_lport_update_port_sec(
>  static bool ovs_iface_matches_lport_iface_id_ver(
>      const struct ovsrec_interface *,
>      const struct sbrec_port_binding *);
> +static bool cleanup_patch_port_local_dps(
> +    const struct sbrec_port_binding *, const struct sbrec_port_binding *cr_pb,
> +    const struct sbrec_port_binding *peer, struct local_datapath *ld,
> +    struct binding_ctx_in *b_ctx_in,
> +    struct binding_ctx_out *b_ctx_out,
> +    bool *cleanup);
> +static bool local_datapath_is_relevant(
> +    struct local_datapath *, struct local_datapath *ignore_peer_ld,
> +    struct hmap *local_datapaths, int *depth, const struct sbrec_chassis *,
> +    struct ovsdb_idl_index *);
>
>  void
>  related_lports_init(struct related_lports *rp)
> @@ -1062,6 +1072,13 @@ binding_dump_related_lports(struct related_lports *related_lports,
>      }
>  }
>
> +struct dp_binding {
> +    struct hmap_node key_node;
> +
> +    uint32_t dp_key;
> +    struct hmapx binding_lports;
> +};

Please ignore this stray struct declaration.  Forgot to clean this up.
I'll remove it in the next version.

Numan

> +
>  void
>  binding_dump_local_bindings(struct local_binding_data *lbinding_data,
>                              struct ds *out_data)
> @@ -1130,6 +1147,19 @@ binding_dump_local_bindings(struct local_binding_data *lbinding_data,
>      free(nodes);
>  }
>
> +void
> +binding_dump_local_datapaths(struct hmap *local_datapaths,
> +                             struct ds *out_data)
> +{
> +    ds_put_cstr(out_data, "Local datapaths:\n");
> +    struct local_datapath *ld;
> +    HMAP_FOR_EACH (ld, hmap_node, local_datapaths) {
> +        ds_put_format(out_data, "Datapath: %s, type: %s\n",
> +                      smap_get(&ld->datapath->external_ids, "name"),
> +                      ld->is_switch ? "switch" : "router");
> +    }
> +}
> +
>  void
>  set_pb_chassis_in_sbrec(const struct sbrec_port_binding *pb,
>                          const struct sbrec_chassis *chassis_rec,
> @@ -2137,7 +2167,9 @@ build_local_bindings(struct binding_ctx_in *b_ctx_in,
>
>  static bool consider_patch_port_for_local_datapaths(
>          const struct sbrec_port_binding *,
> -        struct binding_ctx_in *, struct binding_ctx_out *);
> +        const struct sbrec_port_binding *cr_pb,
> +        struct binding_ctx_in *, struct binding_ctx_out *,
> +        bool check_and_remove_localdps);
>
>  void
>  binding_run(struct binding_ctx_in *b_ctx_in, struct binding_ctx_out *b_ctx_out)
> @@ -2182,7 +2214,8 @@ binding_run(struct binding_ctx_in *b_ctx_in, struct binding_ctx_out *b_ctx_out)
>          switch (lport_type) {
>          case LP_PATCH:
>              update_related_lport(pb, b_ctx_out);
> -            consider_patch_port_for_local_datapaths(pb, b_ctx_in, b_ctx_out);
> +            consider_patch_port_for_local_datapaths(pb, NULL, b_ctx_in,
> +                                                    b_ctx_out, false);
>              break;
>
>          case LP_VTEP:
> @@ -2238,6 +2271,9 @@ binding_run(struct binding_ctx_in *b_ctx_in, struct binding_ctx_out *b_ctx_out)
>              struct lport *lnet_lport = xmalloc(sizeof *lnet_lport);
>              lnet_lport->pb = pb;
>              ovs_list_push_back(&localnet_lports, &lnet_lport->list_node);
> +            if (pb->chassis == b_ctx_in->chassis_rec) {
> +                sbrec_port_binding_set_chassis(pb, NULL);
> +            }
>              break;
>          }
>
> @@ -2566,7 +2602,6 @@ consider_iface_release(const struct ovsrec_interface *iface_rec,
>                                             lbinding->iface->name,
>                                             &lbinding->iface->header_.uuid);
>              }
> -
>          } else if (b_lport && b_lport->type == LP_LOCALPORT) {
>              /* lbinding is associated with a localport.  Remove it from the
>               * related lports. */
> @@ -2956,12 +2991,27 @@ handle_updated_vif_lport(const struct sbrec_port_binding *pb,
>
>  static bool
>  consider_patch_port_for_local_datapaths(const struct sbrec_port_binding *pb,
> +                                        const struct sbrec_port_binding *cr_pb,
>                                          struct binding_ctx_in *b_ctx_in,
> -                                        struct binding_ctx_out *b_ctx_out)
> +                                        struct binding_ctx_out *b_ctx_out,
> +                                        bool check_and_remove_localdps)
>  {
> -    struct local_datapath *ld =
> -        get_local_datapath(b_ctx_out->local_datapaths,
> -                           pb->datapath->tunnel_key);
> +    const struct sbrec_port_binding *peer;
> +    struct local_datapath *peer_ld = NULL;
> +    struct local_datapath *ld = NULL;
> +
> +    ld = get_local_datapath(b_ctx_out->local_datapaths,
> +                            pb->datapath->tunnel_key);
> +    if (ld && ld->has_only_dgp_peer_ports) {
> +        /* Nothing much to do. */
> +        return true;
> +    }
> +
> +    peer = lport_get_peer(pb, b_ctx_in->sbrec_port_binding_by_name);
> +    if (peer) {
> +        peer_ld = get_local_datapath(b_ctx_out->local_datapaths,
> +                                     peer->datapath->tunnel_key);
> +    }
>
>      if (!ld) {
>          /* If 'ld' for this lport is not present, then check if
> @@ -2969,17 +3019,9 @@ consider_patch_port_for_local_datapaths(const struct sbrec_port_binding *pb,
>           * and peer's datapath is already in the local datapaths,
>           * then add this lport's datapath to the local_datapaths.
>           * */
> -        const struct sbrec_port_binding *peer;
> -        struct local_datapath *peer_ld = NULL;
> -        peer = lport_get_peer(pb, b_ctx_in->sbrec_port_binding_by_name);
> -        if (peer) {
> -            peer_ld =
> -                get_local_datapath(b_ctx_out->local_datapaths,
> -                                   peer->datapath->tunnel_key);
> -        }
> -        if (peer_ld && need_add_peer_to_local(
> -                b_ctx_in->sbrec_port_binding_by_name, peer,
> -                b_ctx_in->chassis_rec)) {
> +        if (peer_ld && !peer_ld->has_only_dgp_peer_ports &&
> +            need_add_peer_to_local(b_ctx_in->sbrec_port_binding_by_name, peer,
> +                                   b_ctx_in->chassis_rec)) {
>              ld = add_local_datapath(
>                      b_ctx_in->sbrec_datapath_binding_by_key,
>                      b_ctx_in->sbrec_port_binding_by_datapath,
> @@ -2992,7 +3034,7 @@ consider_patch_port_for_local_datapaths(const struct sbrec_port_binding *pb,
>          /* Add the peer datapath to the local datapaths if it's
>           * not present yet.
>           */
> -        if (need_add_peer_to_local(
> +        if (peer && need_add_peer_to_local(
>                  b_ctx_in->sbrec_port_binding_by_name, pb,
>                  b_ctx_in->chassis_rec)) {
>              add_local_datapath_peer_port(
> @@ -3003,6 +3045,18 @@ consider_patch_port_for_local_datapaths(const struct sbrec_port_binding *pb,
>                  ld, b_ctx_out->local_datapaths,
>                  b_ctx_out->tracked_dp_bindings);
>          }
> +
> +        if (check_and_remove_localdps) {
> +            bool cleanedup = false;
> +            if (!cleanup_patch_port_local_dps(pb, cr_pb, peer, ld, b_ctx_in,
> +                                              b_ctx_out, &cleanedup)) {
> +                return false;
> +            }
> +
> +            if (cleanedup) {
> +                ld = NULL;
> +            }
> +        }
>      }
>
>      /* If this chassis is requested - try to claim. */
> @@ -3021,12 +3075,10 @@ consider_patch_port_for_local_datapaths(const struct sbrec_port_binding *pb,
>          || if_status_is_port_claimed(b_ctx_out->if_mgr, pb->logical_port)) {
>
>          remove_local_lports(pb->logical_port, b_ctx_out);
> -        if (!release_lport(pb, ld, b_ctx_in->chassis_rec,
> -                           !b_ctx_in->ovnsb_idl_txn,
> -                           b_ctx_out->tracked_dp_bindings,
> -                           b_ctx_out->if_mgr)) {
> -            return false;
> -        }
> +        return release_lport(pb, ld, b_ctx_in->chassis_rec,
> +                             !b_ctx_in->ovnsb_idl_txn,
> +                             b_ctx_out->tracked_dp_bindings,
> +                             b_ctx_out->if_mgr);
>      }
>      return true;
>  }
> @@ -3086,8 +3138,8 @@ handle_updated_port(struct binding_ctx_in *b_ctx_in,
>
>      case LP_PATCH:
>          update_related_lport(pb, b_ctx_out);
> -        handled = consider_patch_port_for_local_datapaths(pb, b_ctx_in,
> -                                                              b_ctx_out);
> +        handled = consider_patch_port_for_local_datapaths(pb, NULL, b_ctx_in,
> +                                                          b_ctx_out, true);
>          break;
>
>      case LP_VTEP:
> @@ -3130,8 +3182,8 @@ handle_updated_port(struct binding_ctx_in *b_ctx_in,
>              break;
>          }
>          handled = consider_patch_port_for_local_datapaths(distributed_pb,
> -                                                          b_ctx_in,
> -                                                          b_ctx_out);
> +                                                          pb, b_ctx_in,
> +                                                          b_ctx_out, true);
>          break;
>
>      case LP_EXTERNAL:
> @@ -3901,3 +3953,142 @@ binding_destroy(void)
>      shash_destroy_free_data(&_qos_ports);
>      sset_clear(&_postponed_ports);
>  }
> +
> +static bool
> +is_patch_pb_chassis_relevant(
> +    const struct sbrec_port_binding *pb,
> +    const struct sbrec_chassis *chassis,
> +    struct ovsdb_idl_index *sbrec_port_binding_by_name)
> +{
> +    if (ha_chassis_group_contains(pb->ha_chassis_group, chassis)) {
> +        return true;
> +    }
> +
> +    const struct sbrec_port_binding *pb_crp =
> +        lport_get_cr_port(sbrec_port_binding_by_name, pb);
> +    if (pb_crp) {
> +        return ha_chassis_group_contains(pb_crp->ha_chassis_group, chassis);
> +    }
> +
> +    return false;
> +}
> +
> +static bool
> +cleanup_patch_port_local_dps(const struct sbrec_port_binding *pb,
> +                             const struct sbrec_port_binding *cr_pb,
> +                             const struct sbrec_port_binding *peer,
> +                             struct local_datapath *ld,
> +                             struct binding_ctx_in *b_ctx_in,
> +                             struct binding_ctx_out *b_ctx_out,
> +                             bool *cleanedup)
> +{
> +    *cleanedup = false;
> +    if (!peer) {
> +        /* Remove 'pb' from the ld's peer ports as it has no peer. */
> +        remove_local_datapath_peer_port(pb, ld,
> +                                        b_ctx_out->local_datapaths);
> +    }
> +
> +    /* We can consider removing the 'ld' of the patch port 'pb' from the
> +     * local datapaths, if all the below conditions are met
> +     *     - 'pb' doesn't have a peer or ld' is a router datapath
> +     *     - if 'pb' is a distributed gateway port (dgp), then
> +     *       its chassisredirect port's ha chassis group doesn't
> +     *       contain our 'chassis rec'
> +     *     - and finally 'ld' is not relevant any more.  See
> +     *       local_datapath_is_relevant() for more details.
> +     *
> +     * Note: If 'ld' can be removed, then all its connected local datapaths
> +     * can also be removed.
> +     *
> +     * For example, if we had sw1-port1 ->  sw1 -> lr1 -> sw2 and if
> +     * sw1-port1 resides on this chassis, and if the link between sw1 and
> +     * lr1 is broken, then we can remove lr1 and sw2 from the
> +     * local_datapaths.
> +     * */
> +
> +    bool consider_ld_for_removal = !peer || !ld->is_switch;
> +    if (consider_ld_for_removal && cr_pb) {
> +        consider_ld_for_removal = !ha_chassis_group_contains(
> +            cr_pb->ha_chassis_group, b_ctx_in->chassis_rec);
> +    }
> +
> +    if (!consider_ld_for_removal) {
> +        return true;
> +    }
> +
> +    int depth = 0;
> +
> +    bool is_relevant = local_datapath_is_relevant(
> +        ld, NULL, b_ctx_out->local_datapaths,
> +        &depth, b_ctx_in->chassis_rec,
> +        b_ctx_in->sbrec_port_binding_by_name);
> +
> +    if (depth >= 100) {
> +        /* datapaths are too deeply nested.  Fall back to recompute. */
> +        return false;
> +    }
> +
> +    if (!is_relevant) {
> +        /* This 'ld' can be removed from the local datapaths as
> +            *   - its a router datapath and
> +            *   - it has no peers locally. */
> +        local_datapath_remove_and_destroy(ld, b_ctx_out->local_datapaths,
> +                                          b_ctx_out->tracked_dp_bindings);
> +        *cleanedup = true;
> +    }
> +
> +    return true;
> +}
> +
> +static bool
> +local_datapath_is_relevant(struct local_datapath *ld,
> +                           struct local_datapath *ignore_peer_ld,
> +                           struct hmap *local_datapaths, int *depth,
> +                           const struct sbrec_chassis *chassis,
> +                           struct ovsdb_idl_index *sbrec_pb_by_name)
> +{
> +    if (!sset_is_empty(&ld->claimed_lports) ||
> +        !shash_is_empty(&ld->external_ports) ||
> +        !shash_is_empty(&ld->multichassis_ports) ||
> +        ld->vtep_port) {
> +        return true;
> +    }
> +
> +    bool relevant = false;
> +
> +    if (*depth >= 100) {
> +        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> +        VLOG_WARN_RL(&rl, "datapaths nested too deep");
> +        return true;
> +    }
> +
> +    for (size_t i = 0; i < ld->n_peer_ports && !relevant; i++) {
> +        const struct sbrec_port_binding *remote = ld->peer_ports[i].remote;
> +        const struct sbrec_port_binding *local = ld->peer_ports[i].local;
> +
> +        if (is_patch_pb_chassis_relevant(local, chassis,
> +                                         sbrec_pb_by_name)) {
> +            return  true;
> +        }
> +
> +        if (is_patch_pb_chassis_relevant(remote, chassis,
> +                                         sbrec_pb_by_name)) {
> +            return  true;
> +        }
> +
> +        struct local_datapath *peer_ld;
> +        uint32_t remote_peer_ld_key;
> +        remote_peer_ld_key = ld->peer_ports[i].remote->datapath->tunnel_key;
> +        peer_ld = get_local_datapath(local_datapaths, remote_peer_ld_key);
> +        if (peer_ld && !peer_ld->has_only_dgp_peer_ports &&
> +            peer_ld != ignore_peer_ld) {
> +            *depth = *depth + 1;
> +            relevant = local_datapath_is_relevant(peer_ld, ld,
> +                                                  local_datapaths, depth,
> +                                                  chassis, sbrec_pb_by_name);
> +        }
> +    }
> +
> +    return relevant;
> +}
> diff --git a/controller/binding.h b/controller/binding.h
> index d13ae36c79..a4346c3e10 100644
> --- a/controller/binding.h
> +++ b/controller/binding.h
> @@ -201,6 +201,8 @@ bool binding_handle_port_binding_changes(struct binding_ctx_in *,
>  void binding_tracked_dp_destroy(struct hmap *tracked_datapaths);
>
>  void binding_dump_local_bindings(struct local_binding_data *, struct ds *);
> +void binding_dump_local_datapaths(struct hmap *local_datapaths,
> +                                  struct ds *out_data);
>
>  void binding_dump_related_lports(struct related_lports *related_lports,
>                                   struct ds *);
> diff --git a/controller/local_data.c b/controller/local_data.c
> index e19b2bf865..bb5740f1d6 100644
> --- a/controller/local_data.c
> +++ b/controller/local_data.c
> @@ -53,6 +53,13 @@ static struct tracked_datapath *tracked_datapath_create(
>
>  static bool datapath_is_switch(const struct sbrec_datapath_binding *);
>  static bool datapath_is_transit_switch(const struct sbrec_datapath_binding *);
> +static bool datapath_has_only_dgp_peer_ports(
> +    const struct sbrec_datapath_binding *);
> +static void local_datapath_remove_and_destroy__(
> +    struct local_datapath *ld,
> +    const struct sbrec_port_binding *ignore_peer_port,
> +    struct hmap *local_datapaths,
> +    struct hmap *tracked_datapaths);
>
>  static uint64_t local_datapath_usage;
>
> @@ -86,6 +93,7 @@ local_datapath_alloc(const struct sbrec_datapath_binding *dp)
>      ld->datapath = dp;
>      ld->is_switch = datapath_is_switch(dp);
>      ld->is_transit_switch = datapath_is_transit_switch(dp);
> +    ld->has_only_dgp_peer_ports = datapath_has_only_dgp_peer_ports(dp);
>      shash_init(&ld->external_ports);
>      shash_init(&ld->multichassis_ports);
>      sset_init(&ld->claimed_lports);
> @@ -132,6 +140,14 @@ local_datapath_destroy(struct local_datapath *ld)
>      free(ld);
>  }
>
> +void local_datapath_remove_and_destroy(struct local_datapath *ld,
> +                                       struct hmap *local_datapaths,
> +                                       struct hmap *tracked_datapaths)
> +{
> +    local_datapath_remove_and_destroy__(ld, NULL, local_datapaths,
> +                                        tracked_datapaths);
> +}
> +
>  /* Checks if pb is running on local gw router or pb is a patch port
>   * and the peer datapath should be added to local datapaths. */
>  bool
> @@ -226,12 +242,12 @@ add_local_datapath_peer_port(
>          get_local_datapath(local_datapaths,
>                             peer->datapath->tunnel_key);
>      if (!peer_ld) {
> -        add_local_datapath__(sbrec_datapath_binding_by_key,
> -                             sbrec_port_binding_by_datapath,
> -                             sbrec_port_binding_by_name, 1,
> -                             peer->datapath, chassis, local_datapaths,
> -                             tracked_datapaths);
> -        return;
> +        peer_ld = add_local_datapath__(sbrec_datapath_binding_by_key,
> +                                       sbrec_port_binding_by_datapath,
> +                                       sbrec_port_binding_by_name, 1,
> +                                       peer->datapath, chassis,
> +                                       local_datapaths,
> +                                       tracked_datapaths);
>      }
>
>      local_datapath_peer_port_add(peer_ld, peer, pb);
> @@ -613,6 +629,17 @@ add_local_datapath__(struct ovsdb_idl_index *sbrec_datapath_binding_by_key,
>                               tracked_datapaths);
>      }
>
> +    if (ld->has_only_dgp_peer_ports) {
> +        /* If this flag is set, it means this 'switch' datapath has
> +         *  - one ore many localnet ports.
> +         *  - all the router ports it is connected to are
> +         *    distributed gateway ports (DGPs).
> +         * There is no need to add the routers of the dgps to
> +         * the local datapaths.
> +         * */
> +        return ld;
> +    }
> +
>      if (depth >= 100) {
>          static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
>          VLOG_WARN_RL(&rl, "datapaths nested too deep");
> @@ -705,6 +732,13 @@ datapath_is_transit_switch(const struct sbrec_datapath_binding *ldp)
>      return smap_get(&ldp->external_ids, "interconn-ts") != NULL;
>  }
>
> +static bool
> +datapath_has_only_dgp_peer_ports(const struct sbrec_datapath_binding *ldp)
> +{
> +    return datapath_is_switch(ldp) &&
> +           smap_get_bool(&ldp->external_ids, "only_dgp_peer_ports", false);
> +}
> +
>  bool
>  lb_is_local(const struct sbrec_load_balancer *sbrec_lb,
>              const struct hmap *local_datapaths)
> @@ -745,3 +779,41 @@ lb_is_local(const struct sbrec_load_balancer *sbrec_lb,
>
>      return false;
>  }
> +
> +static void
> +local_datapath_remove_and_destroy__(struct local_datapath *ld,
> +                                    const struct sbrec_port_binding *ignore_pb,
> +                                    struct hmap *local_datapaths,
> +                                    struct hmap *tracked_datapaths)
> +{
> +    for (size_t i = 0; i < ld->n_peer_ports; i++) {
> +        const struct sbrec_port_binding *remote = ld->peer_ports[i].remote;
> +        const struct sbrec_port_binding *local = ld->peer_ports[i].local;
> +
> +        if (local == ignore_pb) {
> +            continue;
> +        }
> +
> +        struct local_datapath *peer_ld;
> +        uint32_t remote_peer_ld_key;
> +
> +        remote_peer_ld_key = ld->peer_ports[i].remote->datapath->tunnel_key;
> +        peer_ld = get_local_datapath(local_datapaths, remote_peer_ld_key);
> +        if (peer_ld && !peer_ld->has_only_dgp_peer_ports) {
> +            local_datapath_remove_and_destroy__(peer_ld, remote,
> +                                                local_datapaths,
> +                                                tracked_datapaths);
> +        } else if (peer_ld && peer_ld->has_only_dgp_peer_ports) {
> +            remove_local_datapath_peer_port(ld->peer_ports[i].remote,
> +                                            peer_ld, local_datapaths);
> +        }
> +    }
> +
> +    hmap_remove(local_datapaths, &ld->hmap_node);
> +    if (tracked_datapaths) {
> +        tracked_datapath_add(ld->datapath, TRACKED_RESOURCE_REMOVED,
> +                             tracked_datapaths);
> +    }
> +
> +    local_datapath_destroy(ld);
> +}
> diff --git a/controller/local_data.h b/controller/local_data.h
> index 857d63a51d..ef01f4259e 100644
> --- a/controller/local_data.h
> +++ b/controller/local_data.h
> @@ -46,6 +46,8 @@ struct local_datapath {
>      const struct sbrec_datapath_binding *datapath;
>      bool is_switch;
>      bool is_transit_switch;
> +    /* Valid only for 'is_switch' local datapath. */
> +    bool has_only_dgp_peer_ports;
>
>      /* The localnet port in this datapath, if any (at most one is allowed). */
>      const struct sbrec_port_binding *localnet_port;
> @@ -91,6 +93,10 @@ struct local_datapath * add_local_datapath(
>
>  void local_datapaths_destroy(struct hmap *local_datapaths);
>  void local_datapath_destroy(struct local_datapath *ld);
> +void local_datapath_remove_and_destroy(struct local_datapath *,
> +                                       struct hmap *local_datapaths,
> +                                       struct hmap *tracked_datapaths);
> +
>  void add_local_datapath_peer_port(
>      const struct sbrec_port_binding *,
>      const struct sbrec_chassis *,
> diff --git a/controller/lport.c b/controller/lport.c
> index 8bc230e896..f164803528 100644
> --- a/controller/lport.c
> +++ b/controller/lport.c
> @@ -102,6 +102,18 @@ lport_get_l3gw_peer(const struct sbrec_port_binding *pb,
>      return get_peer_lport(pb, sbrec_port_binding_by_name);
>  }
>
> +const struct sbrec_port_binding *
> +lport_get_cr_port(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> +                  const struct sbrec_port_binding *pb)
> +{
> +    const char *crp = smap_get(&pb->options, "chassis-redirect-port");
> +    if (crp) {
> +        return lport_lookup_by_name(sbrec_port_binding_by_name, crp);
> +    }
> +
> +    return NULL;
> +}
> +
>  enum can_bind
>  lport_can_bind_on_this_chassis(const struct sbrec_chassis *chassis_rec,
>                                 const struct sbrec_port_binding *pb)
> diff --git a/controller/lport.h b/controller/lport.h
> index 3a58ebe39e..069bd59096 100644
> --- a/controller/lport.h
> +++ b/controller/lport.h
> @@ -73,4 +73,8 @@ const struct sbrec_port_binding *lport_get_l3gw_peer(
>  bool
>  lport_is_activated_by_activation_strategy(const struct sbrec_port_binding *pb,
>                                            const struct sbrec_chassis *chassis);
> +const struct sbrec_port_binding *lport_get_cr_port(
> +    struct ovsdb_idl_index *sbrec_port_binding_by_name,
> +    const struct sbrec_port_binding *);
> +
>  #endif /* controller/lport.h */
> diff --git a/controller/ovn-controller.c b/controller/ovn-controller.c
> index da942abaa9..dc8ceec076 100644
> --- a/controller/ovn-controller.c
> +++ b/controller/ovn-controller.c
> @@ -101,6 +101,7 @@ static unixctl_cb_func debug_pause_execution;
>  static unixctl_cb_func debug_resume_execution;
>  static unixctl_cb_func debug_status_execution;
>  static unixctl_cb_func debug_dump_local_bindings;
> +static unixctl_cb_func debug_dump_local_datapaths;
>  static unixctl_cb_func debug_dump_related_lports;
>  static unixctl_cb_func debug_dump_local_template_vars;
>  static unixctl_cb_func debug_dump_local_mac_bindings;
> @@ -1688,6 +1689,22 @@ runtime_data_sb_datapath_binding_handler(struct engine_node *node OVS_UNUSED,
>                  return false;
>              }
>          }
> +
> +        if (sbrec_datapath_binding_is_updated(
> +                dp, SBREC_DATAPATH_BINDING_COL_EXTERNAL_IDS) &&
> +            !sbrec_datapath_binding_is_new(dp)) {
> +            struct local_datapath *ld =
> +                get_local_datapath(&rt_data->local_datapaths,
> +                                   dp->tunnel_key);
> +                if (ld && ld->is_switch) {
> +                    bool only_dgp_peer_ports =
> +                        smap_get_bool(&dp->external_ids, "only_dgp_peer_ports",
> +                                      false);
> +                    if (ld->has_only_dgp_peer_ports != only_dgp_peer_ports) {
> +                        return false;
> +                    }
> +                }
> +        }
>      }
>
>      return true;
> @@ -4357,6 +4374,12 @@ lflow_output_runtime_data_handler(struct engine_node *node,
>      init_lflow_ctx(node, fo, &l_ctx_in, &l_ctx_out);
>
>      struct tracked_datapath *tdp;
> +    HMAP_FOR_EACH (tdp, node, tracked_dp_bindings) {
> +        if (tdp->tracked_type == TRACKED_RESOURCE_REMOVED) {
> +            return false;
> +        }
> +    }
> +
>      HMAP_FOR_EACH (tdp, node, tracked_dp_bindings) {
>          if (tdp->tracked_type == TRACKED_RESOURCE_NEW) {
>              if (!lflow_add_flows_for_datapath(tdp->dp, &l_ctx_in,
> @@ -5552,6 +5575,10 @@ main(int argc, char *argv[])
>                               debug_dump_local_bindings,
>                               &runtime_data->lbinding_data);
>
> +    unixctl_command_register("debug/dump-local-datapaths", "", 0, 0,
> +                             debug_dump_local_datapaths,
> +                             &runtime_data->local_datapaths);
> +
>      unixctl_command_register("debug/dump-related-ports", "", 0, 0,
>                               debug_dump_related_lports,
>                               &runtime_data->related_lports);
> @@ -6517,6 +6544,17 @@ debug_dump_local_bindings(struct unixctl_conn *conn, int argc OVS_UNUSED,
>      ds_destroy(&binding_data);
>  }
>
> +static void
> +debug_dump_local_datapaths(struct unixctl_conn *conn, int argc OVS_UNUSED,
> +                           const char *argv[] OVS_UNUSED,
> +                           void *local_datapaths)
> +{
> +    struct ds local_dps_data = DS_EMPTY_INITIALIZER;
> +    binding_dump_local_datapaths(local_datapaths, &local_dps_data);
> +    unixctl_command_reply(conn, ds_cstr(&local_dps_data));
> +    ds_destroy(&local_dps_data);
> +}
> +
>  static void
>  debug_dump_related_lports(struct unixctl_conn *conn, int argc OVS_UNUSED,
>                            const char *argv[] OVS_UNUSED, void *related_lports)
> diff --git a/tests/multinode.at b/tests/multinode.at
> index c1bd3123ac..b828f16135 100644
> --- a/tests/multinode.at
> +++ b/tests/multinode.at
> @@ -2776,3 +2776,181 @@ for i in 1 2; do
>  done
>
>  AT_CLEANUP
> +
> +AT_SETUP([ovn multinode - only_dgp_peer_ports provider switch functionality])
> +
> +# Check that ovn-fake-multinode setup is up and running
> +check_fake_multinode_setup
> +
> +# Delete the multinode NB and OVS resources before starting the test.
> +cleanup_multinode_resources
> +
> +check multinode_nbctl ls-add sw0
> +check multinode_nbctl lsp-add sw0 sw0-port1
> +check multinode_nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03 10.0.0.3 1000::3"
> +check multinode_nbctl lsp-add sw0 sw0-port2
> +check multinode_nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:04 10.0.0.4 1000::4"
> +
> +check multinode_nbctl lr-add lr0
> +check multinode_nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 1000::1/64
> +check multinode_nbctl lsp-add sw0 sw0-lr0
> +check multinode_nbctl lsp-set-type sw0-lr0 router
> +check multinode_nbctl lsp-set-addresses sw0-lr0 router
> +check multinode_nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> +
> +check multinode_nbctl ls-add public
> +check multinode_nbctl lrp-add lr0 lr0-public 00:00:20:20:12:13 172.16.1.100/24 2000::1/64
> +check multinode_nbctl lsp-add public public-lr0
> +check multinode_nbctl lsp-set-type public-lr0 router
> +check multinode_nbctl lsp-set-addresses public-lr0 router
> +check multinode_nbctl lsp-set-options public-lr0 router-port=lr0-public
> +
> +# localnet port
> +check multinode_nbctl lsp-add public ln-public
> +check multinode_nbctl lsp-set-type ln-public localnet
> +check multinode_nbctl lsp-set-addresses ln-public unknown
> +check multinode_nbctl lsp-set-options ln-public network_name=public
> +
> +check multinode_nbctl lrp-set-gateway-chassis lr0-public ovn-gw-1 20
> +check multinode_nbctl lr-nat-add lr0 snat 172.16.1.100 10.0.0.0/24
> +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.16.1.110 10.0.0.3 sw0-port1 50:54:00:00:00:03
> +check multinode_nbctl lr-nat-add lr0 snat 2000::1 1000::/64
> +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 2000::2 1000::3 sw0-port1 50:54:00:00:00:03
> +
> +check multinode_nbctl --wait=hv sync
> +
> +check multinode_nbctl ls-add sw1
> +check multinode_nbctl lsp-add sw1 sw1-port1
> +check multinode_nbctl lsp-set-addresses sw1-port1 "40:54:00:00:00:03 20.0.0.3 2000::3"
> +
> +check multinode_nbctl lr-add lr1
> +check multinode_nbctl lrp-add lr1 lr1-sw1 00:00:01:00:ef:01 20.0.0.1/24 2000::a/64
> +check multinode_nbctl lsp-add sw1 sw1-lr1
> +check multinode_nbctl lsp-set-type sw1-lr1 router
> +check multinode_nbctl lsp-set-addresses sw1-lr1 router
> +check multinode_nbctl lsp-set-options sw1-lr1 router-port=lr1-sw1
> +
> +check multinode_nbctl lrp-add lr1 lr1-public 00:00:20:30:22:13 172.16.1.101/24
> +check multinode_nbctl lsp-add public public-lr1
> +check multinode_nbctl lsp-set-type public-lr1 router
> +check multinode_nbctl lsp-set-addresses public-lr1 router
> +check multinode_nbctl lsp-set-options public-lr1 router-port=lr1-public
> +
> +check multinode_nbctl lr-nat-add lr1 snat 172.16.1.101 20.0.0.0/24
> +check multinode_nbctl lr-nat-add lr1 dnat_and_snat 172.16.1.120 20.0.0.3
> +check multinode_nbctl lrp-set-gateway-chassis lr1-public ovn-gw-1 20
> +
> +check multinode_nbctl --wait=hv sync
> +
> +m_as ovn-chassis-1 /data/create_fake_vm.sh sw0-port1 sw0p1 50:54:00:00:00:03 1342 10.0.0.3 24 10.0.0.1 1000::3/64 1000::a
> +m_as ovn-chassis-2 /data/create_fake_vm.sh sw1-port1 sw1p1 40:54:00:00:00:03 1342 20.0.0.3 24 20.0.0.1 2000::4/64 1000::a
> +
> +m_wait_for_ports_up sw0-port1
> +m_wait_for_ports_up sw1-port1
> +
> +m_as ovn-central-az1-1 ovn-sbctl show
> +
> +m_as ovn-chassis-1 ovn-appctl  debug/dump-local-datapaths | sort
> +m_as ovn-chassis-2 ovn-appctl  debug/dump-local-datapaths | sort
> +
> +AT_CHECK([m_as ovn-chassis-1 ovn-appctl  debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([m_as ovn-chassis-2 ovn-appctl  debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: sw1, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([m_as ovn-gw-1 ovn-appctl  debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: lr1, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Datapath: sw1, type: switch
> +Local datapaths:
> +])
> +
> +# ping lr0-public IP - 172.168.0.100
> +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.16.1.100 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +# ping lr1-public IP - 172.168.0.101 from sw0p1
> +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.16.1.101 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +# ping public ip of sw1-port1 - 172.16.1.120 from sw0p1
> +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.16.1.120 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +# ping public ip of sw0-port1 - 172.16.1.110 from sw1p1
> +M_NS_CHECK_EXEC([ovn-chassis-2], [sw1p1], [ping -q -c 3 -i 0.3 -w 2 172.16.1.110 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +# Bind sw0-port2 on chassis-2
> +m_as ovn-chassis-2 /data/create_fake_vm.sh sw0-port2 sw0p2 50:54:00:00:00:04 1342 10.0.0.4 24 10.0.0.1 1000::4/64 1000::a
> +m_wait_for_ports_up sw0-port2
> +
> +AT_CHECK([m_as ovn-chassis-2 ovn-appctl  debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: lr1, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Datapath: sw1, type: switch
> +Local datapaths:
> +])
> +
> +# ping public ip of sw0-port1 - 172.16.1.110 from sw0p2
> +M_NS_CHECK_EXEC([ovn-chassis-2], [sw0p2], [ping -q -c 3 -i 0.3 -w 2 172.16.1.110 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +# ping public ip of sw1-port1 - 172.16.1.120 from sw0p2
> +M_NS_CHECK_EXEC([ovn-chassis-2], [sw0p2], [ping -q -c 3 -i 0.3 -w 2 172.16.1.120 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +# Create a normal router port in public with its peer as a normal distributed router port.
> +check multinode_nbctl lsp-add public public-lr2
> +check multinode_nbctl lsp-set-type public-lr2 router
> +check multinode_nbctl lsp-set-addresses public-lr2 router
> +check multinode_nbctl lsp-set-options public-lr2 router-port=lr2-public
> +check multinode_nbctl lr-add lr2
> +check multinode_nbctl lrp-add lr2 lr2-public 00:00:41:00:1f:61 172.16.1.102/24 3000::a/64
> +
> +check multinode_nbctl --wait=hv sync
> +AT_CHECK([m_as ovn-chassis-1 ovn-appctl  debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Datapath: sw1, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([m_as ovn-chassis-2 ovn-appctl  debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Datapath: sw1, type: switch
> +Local datapaths:
> +])
> +
> +AT_CLEANUP
> diff --git a/tests/ovn-performance.at b/tests/ovn-performance.at
> index 7d480c20c8..a003fc36bc 100644
> --- a/tests/ovn-performance.at
> +++ b/tests/ovn-performance.at
> @@ -479,7 +479,7 @@ OVN_CONTROLLER_EXPECT_NO_HIT(
>  )
>
>  OVN_CONTROLLER_EXPECT_HIT_COND(
> -    [hv1 hv2 hv3 hv4 hv5], [lflow_run], [=0 =0 >0 =0 =0],
> +    [hv1 hv2 hv3 hv4 hv5], [lflow_run], [>0 >0 >0 =0 =0],
>      [ovn-nbctl --wait=hv lrp-set-gateway-chassis lr1-public hv3 30 && ovn-nbctl --wait=hv sync]
>  )
>
> @@ -552,8 +552,8 @@ hv5_ch=$(ovn-sbctl --bare --columns _uuid list chassis hv5)
>  OVS_WAIT_UNTIL([ovn-sbctl find port_binding logical_port=cr-lr1-public chassis=$hv5_ch])
>  check ovn-nbctl --wait=hv sync
>  # Delete hv5 from gateway chassis. There should be no lflow_run.
> -OVN_CONTROLLER_EXPECT_NO_HIT(
> -    [hv1 hv2 hv3 hv4 hv5], [lflow_run],
> +OVN_CONTROLLER_EXPECT_HIT_COND(
> +    [hv1 hv2 hv3 hv4 hv5], [lflow_run], [=0 =0 =0 =0 =0]
>      [ovn-nbctl --wait=hv lrp-del-gateway-chassis lr1-public hv5]
>  )
>
> diff --git a/tests/ovn.at b/tests/ovn.at
> index d105ed2535..d4e87212a5 100644
> --- a/tests/ovn.at
> +++ b/tests/ovn.at
> @@ -41481,6 +41481,859 @@ OVN_CLEANUP([hv1])
>  AT_CLEANUP
>  ])
>
> +OVN_FOR_EACH_NORTHD_NO_HV([
> +AT_SETUP([ovn-controller -- only_dgp_peer_ports flag in SB datapath_binding])
> +AT_KEYWORDS([multiple-l3dgw-ports])
> +ovn_start
> +net_add n1
> +sim_add hv1
> +as hv1
> +check ovs-vsctl add-br br-phys
> +ovn_attach n1 br-phys 192.168.0.1
> +
> +sim_add hv2
> +as hv2
> +check ovs-vsctl add-br br-phys
> +ovn_attach n1 br-phys 192.168.0.2
> +
> +sim_add gw1
> +as gw1
> +check ovs-vsctl add-br br-phys
> +ovn_attach n1 br-phys 192.168.0.3
> +
> +sim_add gw2
> +as gw2
> +check ovs-vsctl add-br br-phys
> +ovn_attach n1 br-phys 192.168.0.4
> +
> +check ovn-nbctl ls-add sw0
> +check ovn-nbctl lsp-add sw0 sw0-port1
> +check ovn-nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:01 10.0.0.3 1000::3"
> +check ovn-nbctl lsp-add sw0 sw0-port2
> +check ovn-nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:02 10.0.0.4 1000::4"
> +
> +check ovn-nbctl lr-add lr0
> +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 1000::1/64
> +check ovn-nbctl lsp-add sw0 sw0-lr0
> +check ovn-nbctl lsp-set-type sw0-lr0 router
> +check ovn-nbctl lsp-set-addresses sw0-lr0 router
> +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> +
> +check ovn-nbctl ls-add public
> +check ovn-nbctl lrp-add lr0 lr0-public 00:00:20:20:12:13 172.168.0.100/24 2000::1/64
> +check ovn-nbctl lsp-add public public-lr0
> +check ovn-nbctl lsp-set-type public-lr0 router
> +check ovn-nbctl lsp-set-addresses public-lr0 router
> +check ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public
> +
> +# localnet port
> +check ovn-nbctl lsp-add public ln-public
> +check ovn-nbctl lsp-set-type ln-public localnet
> +check ovn-nbctl lsp-set-addresses ln-public unknown
> +check ovn-nbctl lsp-set-options ln-public network_name=phys
> +
> +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1 20
> +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 10.0.0.0/24
> +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 sw0-port2 f0:00:00:01:02:04
> +check ovn-nbctl lr-nat-add lr0 snat 2000::1 1000::/64
> +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 2000::2 1000::4 sw0-port2 f0:00:00:01:02:04
> +
> +check ovn-nbctl --wait=hv sync
> +
> +sw0_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=sw0))
> +lr0_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=lr0))
> +public_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=public))
> +
> +check_offlows_for_datapath() {
> +    hv=$1
> +    dp_key=$2
> +    should_be_present=$3
> +
> +    if [[ "$should_be_present" == "yes" ]]; then
> +        echo "Flows should be present for hv - $hv : datapath - $dp_key"
> +        OVS_WAIT_UNTIL(
> +            [test $(as $hv ovs-ofctl dump-flows br-int | grep -c metadata=0x$dp_key) -gt 0]
> +        )
> +    else
> +        echo "Flows should NOT be present for hv - $hv : datapath - $dp_key"
> +        OVS_WAIT_UNTIL(
> +            [test $(as $hv ovs-ofctl dump-flows br-int | grep -c metadata=0x$dp_key) -eq 0]
> +        )
> +    fi
> +}
> +
> +AT_CHECK([ovn-sbctl get datapath_binding public external_ids:only_dgp_peer_ports], [0], [dnl
> +"true"
> +])
> +
> +check_offlows_for_datapath hv1 $sw0_dp_key no
> +check_offlows_for_datapath hv1 $lr0_dp_key no
> +check_offlows_for_datapath hv1 $public_dp_key no
> +
> +check_offlows_for_datapath hv2 $sw0_dp_key no
> +check_offlows_for_datapath hv2 $lr0_dp_key no
> +check_offlows_for_datapath hv2 $public_dp_key no
> +
> +check_offlows_for_datapath gw1 $sw0_dp_key yes
> +check_offlows_for_datapath gw1 $lr0_dp_key yes
> +check_offlows_for_datapath gw1 $public_dp_key yes
> +
> +check_offlows_for_datapath gw2 $sw0_dp_key no
> +check_offlows_for_datapath gw2 $lr0_dp_key no
> +check_offlows_for_datapath gw2 $public_dp_key no
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> +Local datapaths:
> +])
> +
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> +Local datapaths:
> +])
> +
> +# Create a VIF on hv1 for sw0-port1
> +AS_BOX([create a VIF on hv1 for sw0-port1])
> +
> +as hv1
> +ovs-vsctl -- add-port br-int hv1-vif1 -- \
> +    set interface hv1-vif1 external-ids:iface-id=sw0-port1 \
> +    options:tx_pcap=hv1/vif1-tx.pcap \
> +    options:rxq_pcap=hv1/vif1-rx.pcap \
> +    ofport-request=1
> +
> +wait_for_ports_up sw0-port1
> +
> +AS_BOX([Create a VIF on hv1 for sw0-port1 - hv1 should have flows for sw0, lr0 and public])
> +
> +check_offlows_for_datapath hv1 $sw0_dp_key yes
> +check_offlows_for_datapath hv1 $lr0_dp_key yes
> +check_offlows_for_datapath hv1 $public_dp_key yes
> +
> +AS_BOX([hv2 should NOT have flows for sw0, lr0 and public])
> +check_offlows_for_datapath hv2 $sw0_dp_key no
> +check_offlows_for_datapath hv2 $lr0_dp_key no
> +check_offlows_for_datapath hv2 $public_dp_key no
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> +Local datapaths:
> +])
> +
> +AS_BOX([create a switch sw1 and router lr1, attach both and attach lr1 to public])
> +
> +check ovn-nbctl ls-add sw1
> +check ovn-nbctl lsp-add sw1 sw1-port1
> +check ovn-nbctl lsp-set-addresses sw1-port1 "60:54:00:00:00:01 20.0.0.3"
> +
> +check ovn-nbctl lr-add lr1
> +check ovn-nbctl lrp-add lr1 lr1-sw1 00:00:01:00:ef:01 20.0.0.1/24
> +check ovn-nbctl lsp-add sw1 sw1-lr1
> +check ovn-nbctl lsp-set-type sw1-lr1 router
> +check ovn-nbctl lsp-set-addresses sw1-lr1 router
> +check ovn-nbctl lsp-set-options sw1-lr1 router-port=lr1-sw1
> +
> +check ovn-nbctl lrp-add lr1 lr1-public 00:00:20:30:22:13 172.168.0.101/24
> +check ovn-nbctl lsp-add public public-lr1
> +check ovn-nbctl lsp-set-type public-lr1 router
> +check ovn-nbctl lsp-set-addresses public-lr1 router
> +check ovn-nbctl lsp-set-options public-lr1 router-port=lr1-public
> +
> +sw1_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=sw1))
> +lr1_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=lr1))
> +
> +check ovn-nbctl lr-nat-add lr1 snat 172.168.0.101 20.0.0.0/24
> +check ovn-nbctl lr-nat-add lr1 dnat_and_snat 172.168.0.140 20.0.0.3
> +
> +AS_BOX([create a switch sw2 and router lr2, attach both and attach lr2 to public])
> +
> +check ovn-nbctl ls-add sw2
> +check ovn-nbctl lsp-add sw2 sw2-port1
> +check ovn-nbctl lsp-set-addresses sw2-port1 "70:54:00:00:00:01 30.0.0.3"
> +
> +check ovn-nbctl lr-add lr2
> +check ovn-nbctl lrp-add lr2 lr2-sw2 00:00:02:00:ef:01 30.0.0.1/24
> +check ovn-nbctl lsp-add sw2 sw2-lr2
> +check ovn-nbctl lsp-set-type sw2-lr2 router
> +check ovn-nbctl lsp-set-addresses sw2-lr2 router
> +check ovn-nbctl lsp-set-options sw2-lr2 router-port=lr2-sw2
> +
> +check ovn-nbctl lrp-add lr2 lr2-public 00:00:20:40:22:53 172.168.0.102/24
> +check ovn-nbctl lsp-add public public-lr2
> +check ovn-nbctl lsp-set-type public-lr2 router
> +check ovn-nbctl lsp-set-addresses public-lr2 router
> +check ovn-nbctl lsp-set-options public-lr2 router-port=lr2-public
> +
> +sw2_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=sw2))
> +lr2_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=lr2))
> +
> +check ovn-nbctl lr-nat-add lr2 snat 172.168.0.102 30.0.0.0/24
> +check ovn-nbctl lr-nat-add lr2 dnat_and_snat 172.168.0.150 30.0.0.3
> +
> +check ovn-nbctl --wait=hv sync
> +
> +# Since lr1-public is not a DGP,  public is not a "only_dgp_peer_ports".
> +AT_CHECK([ovn-sbctl get datapath_binding public external_ids:only_dgp_peer_ports], [1], [ignore], [ignore])
> +
> +check_offlows_for_datapath hv1 $sw1_dp_key yes
> +check_offlows_for_datapath hv1 $lr1_dp_key yes
> +check_offlows_for_datapath hv1 $sw2_dp_key yes
> +check_offlows_for_datapath hv1 $lr2_dp_key yes
> +
> +check_offlows_for_datapath hv2 $sw1_dp_key no
> +check_offlows_for_datapath hv2 $lr1_dp_key no
> +check_offlows_for_datapath hv2 $sw2_dp_key no
> +check_offlows_for_datapath hv2 $lr2_dp_key no
> +
> +check_offlows_for_datapath gw1 $sw1_dp_key yes
> +check_offlows_for_datapath gw1 $lr1_dp_key yes
> +check_offlows_for_datapath gw1 $sw2_dp_key yes
> +check_offlows_for_datapath gw1 $lr2_dp_key yes
> +
> +check_offlows_for_datapath gw2 $sw1_dp_key no
> +check_offlows_for_datapath gw2 $lr1_dp_key no
> +check_offlows_for_datapath gw2 $sw2_dp_key no
> +check_offlows_for_datapath gw2 $lr2_dp_key no
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Datapath: sw1, type: switch
> +Datapath: sw2, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Datapath: sw1, type: switch
> +Datapath: sw2, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> +Local datapaths:
> +])
> +
> +AS_BOX([Set gw2 as gateway chassis for lr1-public and lr2-public])
> +check ovn-nbctl --wait=hv lrp-set-gateway-chassis lr1-public gw2 20
> +check ovn-nbctl --wait=hv lrp-set-gateway-chassis lr2-public gw2 30
> +wait_row_count Port_Binding 1 logical_port=cr-lr1-public
> +wait_row_count Port_Binding 1 logical_port=cr-lr2-public
> +
> +AT_CHECK([ovn-sbctl get datapath_binding public external_ids:only_dgp_peer_ports], [0], [dnl
> +"true"
> +])
> +
> +check ovn-nbctl --wait=hv sync
> +
> +check_offlows_for_datapath hv1 $sw0_dp_key yes
> +check_offlows_for_datapath hv1 $lr0_dp_key yes
> +check_offlows_for_datapath hv1 $public_dp_key yes
> +check_offlows_for_datapath hv1 $sw1_dp_key no
> +check_offlows_for_datapath hv1 $lr1_dp_key no
> +check_offlows_for_datapath hv1 $sw2_dp_key no
> +check_offlows_for_datapath hv1 $lr2_dp_key no
> +
> +check_offlows_for_datapath hv2 $sw0_dp_key no
> +check_offlows_for_datapath hv2 $lr0_dp_key no
> +check_offlows_for_datapath hv2 $public_dp_key no
> +check_offlows_for_datapath hv2 $sw1_dp_key no
> +check_offlows_for_datapath hv2 $lr1_dp_key no
> +check_offlows_for_datapath hv2 $sw2_dp_key no
> +check_offlows_for_datapath hv2 $lr2_dp_key no
> +
> +check_offlows_for_datapath gw1 $sw0_dp_key yes
> +check_offlows_for_datapath gw1 $lr0_dp_key yes
> +check_offlows_for_datapath gw1 $public_dp_key yes
> +check_offlows_for_datapath gw1 $sw1_dp_key no
> +check_offlows_for_datapath gw1 $lr1_dp_key no
> +check_offlows_for_datapath gw1 $sw2_dp_key no
> +check_offlows_for_datapath gw1 $lr2_dp_key no
> +
> +check_offlows_for_datapath gw2 $sw0_dp_key no
> +check_offlows_for_datapath gw2 $lr0_dp_key no
> +check_offlows_for_datapath hv1 $public_dp_key yes
> +check_offlows_for_datapath gw2 $sw1_dp_key yes
> +check_offlows_for_datapath gw2 $lr1_dp_key yes
> +check_offlows_for_datapath gw2 $sw2_dp_key yes
> +check_offlows_for_datapath gw2 $lr2_dp_key yes
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw1, type: switch
> +Datapath: sw2, type: switch
> +Local datapaths:
> +])
> +
> +AS_BOX([Create a VIF on hv2 for sw1-port1])
> +
> +as hv2
> +ovs-vsctl -- add-port br-int hv2-vif1 -- \
> +    set interface hv2-vif1 external-ids:iface-id=sw1-port1 \
> +    options:tx_pcap=hv2/vif1-tx.pcap \
> +    options:rxq_pcap=hv2/vif1-rx.pcap \
> +    ofport-request=1
> +
> +wait_for_ports_up sw1-port1
> +
> +check_offlows_for_datapath hv1 $sw0_dp_key yes
> +check_offlows_for_datapath hv1 $lr0_dp_key yes
> +check_offlows_for_datapath hv1 $public_dp_key yes
> +check_offlows_for_datapath hv1 $sw1_dp_key no
> +check_offlows_for_datapath hv1 $lr1_dp_key no
> +check_offlows_for_datapath hv1 $sw2_dp_key no
> +check_offlows_for_datapath hv1 $lr2_dp_key no
> +
> +check_offlows_for_datapath hv2 $sw0_dp_key no
> +check_offlows_for_datapath hv2 $lr0_dp_key no
> +
> +# Since there are no distributed dnat_and_snat entries
> +# in lr1, hv2 will not have "public" in its
> +# local datapaths.
> +check_offlows_for_datapath hv2 $public_dp_key no
> +check_offlows_for_datapath hv2 $sw1_dp_key yes
> +check_offlows_for_datapath hv2 $lr1_dp_key yes
> +check_offlows_for_datapath hv2 $sw2_dp_key no
> +check_offlows_for_datapath hv2 $lr2_dp_key no
> +
> +check_offlows_for_datapath gw1 $sw0_dp_key yes
> +check_offlows_for_datapath gw1 $lr0_dp_key yes
> +check_offlows_for_datapath gw1 $public_dp_key yes
> +check_offlows_for_datapath gw1 $sw1_dp_key no
> +check_offlows_for_datapath gw1 $lr1_dp_key no
> +check_offlows_for_datapath gw1 $sw2_dp_key no
> +check_offlows_for_datapath gw1 $lr2_dp_key no
> +
> +# gw2 should have sw1, lr1, sw2 and lr2 and public in its local datapaths.
> +check_offlows_for_datapath gw2 $sw0_dp_key no
> +check_offlows_for_datapath gw2 $lr0_dp_key no
> +check_offlows_for_datapath gw2 $public_dp_key yes
> +check_offlows_for_datapath gw2 $sw1_dp_key yes
> +check_offlows_for_datapath gw2 $lr1_dp_key yes
> +check_offlows_for_datapath gw2 $sw2_dp_key yes
> +check_offlows_for_datapath gw2 $lr2_dp_key yes
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: sw1, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw1, type: switch
> +Datapath: sw2, type: switch
> +Local datapaths:
> +])
> +
> +# Add distributed dnat_and_snat in lr1.  hv2 should have
> +# public in its local datapaths.
> +AS_BOX([ Add distributed dnat_and_snat in lr1])
> +
> +check ovn-nbctl lr-nat-del lr1 dnat_and_snat
> +check ovn-nbctl --wait=hv lr-nat-add lr1 dnat_and_snat 172.168.0.140 20.0.0.3 sw1-port1 10:00:00:01:02:14
> +
> +check_offlows_for_datapath hv2 $sw0_dp_key no
> +check_offlows_for_datapath hv2 $lr0_dp_key no
> +check_offlows_for_datapath hv2 $public_dp_key yes
> +check_offlows_for_datapath hv2 $sw1_dp_key yes
> +check_offlows_for_datapath hv2 $lr1_dp_key yes
> +check_offlows_for_datapath hv2 $sw2_dp_key no
> +check_offlows_for_datapath hv2 $lr2_dp_key no
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: public, type: switch
> +Datapath: sw1, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw1, type: switch
> +Datapath: sw2, type: switch
> +Local datapaths:
> +])
> +
> +AS_BOX([Create a VIF on hv2 for sw0-port2])
> +
> +as hv2
> +ovs-vsctl -- add-port br-int hv2-vif2 -- \
> +    set interface hv2-vif2 external-ids:iface-id=sw0-port2 \
> +    options:tx_pcap=hv2/vif2-tx.pcap \
> +    options:rxq_pcap=hv2/vif2-rx.pcap \
> +    ofport-request=2
> +
> +wait_for_ports_up sw0-port2
> +
> +check_offlows_for_datapath hv2 $sw0_dp_key yes
> +check_offlows_for_datapath hv2 $lr0_dp_key yes
> +check_offlows_for_datapath hv2 $public_dp_key yes
> +check_offlows_for_datapath hv2 $sw1_dp_key yes
> +check_offlows_for_datapath hv2 $lr1_dp_key yes
> +check_offlows_for_datapath hv2 $sw2_dp_key no
> +check_offlows_for_datapath hv2 $lr2_dp_key no
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: lr1, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Datapath: sw1, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw1, type: switch
> +Datapath: sw2, type: switch
> +Local datapaths:
> +])
> +
> +AS_BOX([Delete the VIF for sw1-port1 in hv2])
> +
> +as hv2 ovs-vsctl del-port hv2-vif1
> +check ovn-nbctl --wait=hv sync
> +check_column "false" Port_Binding up logical_port=sw1-port1
> +
> +check_offlows_for_datapath hv2 $sw0_dp_key yes
> +check_offlows_for_datapath hv2 $lr0_dp_key yes
> +check_offlows_for_datapath hv2 $public_dp_key yes
> +check_offlows_for_datapath hv2 $sw1_dp_key no
> +check_offlows_for_datapath hv2 $lr1_dp_key no
> +check_offlows_for_datapath hv2 $sw2_dp_key no
> +check_offlows_for_datapath hv2 $lr2_dp_key no
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw1, type: switch
> +Datapath: sw2, type: switch
> +Local datapaths:
> +])
> +
> +AS_BOX([Delete the VIF for sw0-port2 in hv2])
> +
> +# Presently when a port binding is released we are not
> +# deleting its datapath from the local_datapaths if it
> +# is not relevant anymore.
> +
> +as hv2 ovs-vsctl del-port hv2-vif2
> +check ovn-nbctl --wait=hv sync
> +check_column "false" Port_Binding up logical_port=sw0-port2
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +# hv2 would still have public, sw0 and lr0 in its local datapaths.
> +# Next recompute should delete these datapaths.
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw1, type: switch
> +Datapath: sw2, type: switch
> +Local datapaths:
> +])
> +
> +# Trigger a recompute
> +AS_BOX([Trigger a recompute in hv2])
> +check as hv2 ovn-appctl inc-engine/recompute
> +
> +check_offlows_for_datapath hv2 $sw0_dp_key no
> +check_offlows_for_datapath hv2 $lr0_dp_key no
> +check_offlows_for_datapath hv2 $public_dp_key no
> +check_offlows_for_datapath hv2 $sw1_dp_key no
> +check_offlows_for_datapath hv2 $lr1_dp_key no
> +check_offlows_for_datapath hv2 $sw2_dp_key no
> +check_offlows_for_datapath hv2 $lr2_dp_key no
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw1, type: switch
> +Datapath: sw2, type: switch
> +Local datapaths:
> +])
> +
> +AS_BOX([Disconnect sw2 from lr2])
> +
> +check ovn-nbctl --wait=hv lsp-set-options sw2-lr2 router-port=lr2-sw2xxx
> +check_offlows_for_datapath hv1 $sw0_dp_key yes
> +check_offlows_for_datapath hv1 $lr0_dp_key yes
> +check_offlows_for_datapath hv1 $public_dp_key yes
> +check_offlows_for_datapath hv1 $sw1_dp_key no
> +check_offlows_for_datapath hv1 $lr1_dp_key no
> +check_offlows_for_datapath hv1 $sw2_dp_key no
> +check_offlows_for_datapath hv1 $lr2_dp_key no
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw1, type: switch
> +Local datapaths:
> +])
> +
> +AS_BOX([Reconnect sw2 to lr2 again])
> +
> +check ovn-nbctl --wait=hv lsp-set-options sw2-lr2 router-port=lr2-sw2
> +check_offlows_for_datapath hv1 $sw0_dp_key yes
> +check_offlows_for_datapath hv1 $lr0_dp_key yes
> +check_offlows_for_datapath hv1 $public_dp_key yes
> +check_offlows_for_datapath hv1 $sw1_dp_key no
> +check_offlows_for_datapath hv1 $lr1_dp_key no
> +check_offlows_for_datapath hv1 $sw2_dp_key no
> +check_offlows_for_datapath hv1 $lr2_dp_key no
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw1, type: switch
> +Datapath: sw2, type: switch
> +Local datapaths:
> +])
> +
> +AS_BOX([Create a VIF on gw2 for sw1-port1])
> +
> +as gw2
> +ovs-vsctl -- add-port br-int gw2-vif2 -- \
> +    set interface gw2-vif2 external-ids:iface-id=sw1-port1 \
> +    options:tx_pcap=gw2/vif2-tx.pcap \
> +    options:rxq_pcap=gw2/vif2-rx.pcap \
> +    ofport-request=2
> +
> +wait_for_ports_up sw1-port1
> +
> +check_offlows_for_datapath gw2 $sw0_dp_key no
> +check_offlows_for_datapath gw2 $lr0_dp_key no
> +check_offlows_for_datapath gw2 $public_dp_key yes
> +check_offlows_for_datapath gw2 $sw1_dp_key yes
> +check_offlows_for_datapath gw2 $lr1_dp_key yes
> +check_offlows_for_datapath gw2 $sw2_dp_key yes
> +check_offlows_for_datapath gw2 $lr2_dp_key yes
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw1, type: switch
> +Datapath: sw2, type: switch
> +Local datapaths:
> +])
> +
> +AS_BOX([Delete the VIF for sw1-port1 in gw2])
> +
> +as gw2 ovs-vsctl del-port gw2-vif2
> +check ovn-nbctl --wait=hv sync
> +check_column "false" Port_Binding up logical_port=sw1-port1
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +# hv2 would still have public in its local datapaths.  Next recompute should
> +# delete this datapath from the local datapaths.
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw1, type: switch
> +Datapath: sw2, type: switch
> +Local datapaths:
> +])
> +
> +AS_BOX([Create a logical port for public and bind it on hv2])
> +# hv2 will only have public in its local datapaths.
> +check ovn-nbctl lsp-add public public-p1
> +
> +as hv2
> +ovs-vsctl -- add-port br-int hv2-vif3 -- \
> +    set interface hv2-vif3 external-ids:iface-id=public-p1 \
> +    options:tx_pcap=hv2/vif3-tx.pcap \
> +    options:rxq_pcap=hv2/vif3-rx.pcap \
> +    ofport-request=2
> +
> +wait_for_ports_up public-p1
> +
> +# as hv2 ovn-appctl -t ovn-controller inc-engine/recompute
> +# check ovn-nbctl --wait=hv sync
> +
> +as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort
> +
> +check_offlows_for_datapath hv2 $sw0_dp_key no
> +check_offlows_for_datapath hv2 $lr0_dp_key no
> +check_offlows_for_datapath hv2 $public_dp_key yes
> +check_offlows_for_datapath hv2 $sw1_dp_key no
> +check_offlows_for_datapath hv2 $lr1_dp_key no
> +check_offlows_for_datapath hv2 $sw2_dp_key no
> +check_offlows_for_datapath hv2 $lr2_dp_key no
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: public, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw1, type: switch
> +Datapath: sw2, type: switch
> +Local datapaths:
> +])
> +
> +OVN_CLEANUP([hv1], [hv2], [gw1], [gw2])
> +AT_CLEANUP
> +])
> +
>  OVN_FOR_EACH_NORTHD([
>  AT_SETUP([requested-tnl-key-recompute])
>  AT_KEYWORDS([requested-tnl-key-recompute])
> --
> 2.48.1
>

On Mon, Feb 10, 2025 at 10:37:25AM -0500, numans@ovn.org wrote:
> From: Numan Siddique <numans@ovn.org>
> 
> Consider the below logical topology
> 
> sw0-p1 -
>         |
> sw0-p2 -   ->  sw0 -> lr0 ----
> ...     |                     |
> sw0-pn -                      |
>                               |
> sw1-p1 -                      |
>         |                     |
> sw1p-2 -   ->  sw1 -> lr1 ----  --- public (provider switch)
> ...     |                     |
> sw1-pn-                       |
>                               |
> swn-p1 -                      |
>         |                     |
> swn-p2-    ->  swn -> lrn ----
> ...     |
> swn-pn -
> 
> All the routers are connected to the provider switch via
> a ditributed gateway port.
> 
> If sw0-p1 is resident on the chassis C1, then since there is a path
> to all the switches and the routers, ovn-controller will add all
> these datapaths to its 'local_datapaths' map.  This in turn results
> in processing all the logical flows and installing all the openflows
> and in turn wasting the CPU time.  This can be very costly in
> a highly scaled deployment.
> 
> Previous commit sets a flag "only_dgp_peer_ports" in the SB Datapath
> binding for a provider switch (with only dgp peer ports).
> 
> In this commit, ovn-controller makes use of this flag and stops
> adding other datapaths connected to the public provider switch
> to the 'local_datapaths'.
> 
> For example, when it claims sw0-p1, it adds sw0, lr0 and public
> to the local_datapaths and stops there.  If it later claims
> sw1-p1, it will add sw1 and lr1.
> 
> This reduces the recompute time and the number of openflow rules
> added to ovs-vswitchd significantly.
> 
> I tested this patch with a deployment of below logical resources:
> 
> No of logical switches - 778
> No of logical routers  - 871
> No of logical flows    - 85626
> No of 'ovn-sbctl dump-flows' - 208631
> 
> Without this patch, afte claiming sw0-p1, ovn-controller adds
> 269098 openflow rules and it takes approx 2500 milli seconds
> for a recompute.
> 
> With this patch, after claiming sw0-p1, ovn-controller adds
> 21350 openflow rules and it takes approx 280 milli seconds
> for a recompute.
> 
> There is approx 90% reduction in the openflow rules and
> 88% reduction in recompute time when a comoute node has
> VIFs from one logical switch.

Hi Numan,

that sounds really great. We have a similar setup that would heavily
benefit from this.

However when reading through the patch i have one concern about the
correctness of not considering these ports, but that might just come
from misunderstanding something.

I assumed that we have one chassis c1 that has claimed sw0-p1. It should
then know sw0, lr0 and public.
I also assumed that there is chassis c2 claiming sw1-p1. That would then
know sw1, lr1 and public.
Then i assumed there is a chassis g1 that has claimed the lr0 CR port.
Also chassis g2 has claimed the lr1 CR port.

Now we add a NAT rule to lr0. It is configured with dnat_and_snat and
the settings necessary to process it in a distributed way (i think by
setting logical_port and stateless, but i may be wrong here). It maps
the sw0-p1 IP to some IP on public.

On lr1 we have a similar setup of NAT entries for the sw1-p1 IP.

If i understand these NAT settins correctly this should now lead to traffic
from sw0-p1 to sw1-p1 to flow directly (when using the external ip of the
sw1-p1 NAT rule). So the traffic would flow from chassic c1 to c2
without any involvement of g1 and g2.

In this case wouldn't chassis c1 need to know about the lr1 and sw1
datapath?

I guess i missed something somewhere, but since i am not sure i wanted
to share this concern.

Thanks a lot,
Felix

> 
> Signed-off-by: Numan Siddique <numans@ovn.org>
> ---
>  controller/binding.c        | 249 +++++++++--
>  controller/binding.h        |   2 +
>  controller/local_data.c     |  84 +++-
>  controller/local_data.h     |   6 +
>  controller/lport.c          |  12 +
>  controller/lport.h          |   4 +
>  controller/ovn-controller.c |  38 ++
>  tests/multinode.at          | 178 ++++++++
>  tests/ovn-performance.at    |   6 +-
>  tests/ovn.at                | 853 ++++++++++++++++++++++++++++++++++++
>  10 files changed, 1394 insertions(+), 38 deletions(-)
> 
> diff --git a/controller/binding.c b/controller/binding.c
> index ea5bf5a9fd..88c9640c3a 100644
> --- a/controller/binding.c
> +++ b/controller/binding.c
> @@ -825,6 +825,16 @@ static bool binding_lport_update_port_sec(
>  static bool ovs_iface_matches_lport_iface_id_ver(
>      const struct ovsrec_interface *,
>      const struct sbrec_port_binding *);
> +static bool cleanup_patch_port_local_dps(
> +    const struct sbrec_port_binding *, const struct sbrec_port_binding *cr_pb,
> +    const struct sbrec_port_binding *peer, struct local_datapath *ld,
> +    struct binding_ctx_in *b_ctx_in,
> +    struct binding_ctx_out *b_ctx_out,
> +    bool *cleanup);
> +static bool local_datapath_is_relevant(
> +    struct local_datapath *, struct local_datapath *ignore_peer_ld,
> +    struct hmap *local_datapaths, int *depth, const struct sbrec_chassis *,
> +    struct ovsdb_idl_index *);
>  
>  void
>  related_lports_init(struct related_lports *rp)
> @@ -1062,6 +1072,13 @@ binding_dump_related_lports(struct related_lports *related_lports,
>      }
>  }
>  
> +struct dp_binding {
> +    struct hmap_node key_node;
> +
> +    uint32_t dp_key;
> +    struct hmapx binding_lports;
> +};
> +
>  void
>  binding_dump_local_bindings(struct local_binding_data *lbinding_data,
>                              struct ds *out_data)
> @@ -1130,6 +1147,19 @@ binding_dump_local_bindings(struct local_binding_data *lbinding_data,
>      free(nodes);
>  }
>  
> +void
> +binding_dump_local_datapaths(struct hmap *local_datapaths,
> +                             struct ds *out_data)
> +{
> +    ds_put_cstr(out_data, "Local datapaths:\n");
> +    struct local_datapath *ld;
> +    HMAP_FOR_EACH (ld, hmap_node, local_datapaths) {
> +        ds_put_format(out_data, "Datapath: %s, type: %s\n",
> +                      smap_get(&ld->datapath->external_ids, "name"),
> +                      ld->is_switch ? "switch" : "router");
> +    }
> +}
> +
>  void
>  set_pb_chassis_in_sbrec(const struct sbrec_port_binding *pb,
>                          const struct sbrec_chassis *chassis_rec,
> @@ -2137,7 +2167,9 @@ build_local_bindings(struct binding_ctx_in *b_ctx_in,
>  
>  static bool consider_patch_port_for_local_datapaths(
>          const struct sbrec_port_binding *,
> -        struct binding_ctx_in *, struct binding_ctx_out *);
> +        const struct sbrec_port_binding *cr_pb,
> +        struct binding_ctx_in *, struct binding_ctx_out *,
> +        bool check_and_remove_localdps);
>  
>  void
>  binding_run(struct binding_ctx_in *b_ctx_in, struct binding_ctx_out *b_ctx_out)
> @@ -2182,7 +2214,8 @@ binding_run(struct binding_ctx_in *b_ctx_in, struct binding_ctx_out *b_ctx_out)
>          switch (lport_type) {
>          case LP_PATCH:
>              update_related_lport(pb, b_ctx_out);
> -            consider_patch_port_for_local_datapaths(pb, b_ctx_in, b_ctx_out);
> +            consider_patch_port_for_local_datapaths(pb, NULL, b_ctx_in,
> +                                                    b_ctx_out, false);
>              break;
>  
>          case LP_VTEP:
> @@ -2238,6 +2271,9 @@ binding_run(struct binding_ctx_in *b_ctx_in, struct binding_ctx_out *b_ctx_out)
>              struct lport *lnet_lport = xmalloc(sizeof *lnet_lport);
>              lnet_lport->pb = pb;
>              ovs_list_push_back(&localnet_lports, &lnet_lport->list_node);
> +            if (pb->chassis == b_ctx_in->chassis_rec) {
> +                sbrec_port_binding_set_chassis(pb, NULL);
> +            }
>              break;
>          }
>  
> @@ -2566,7 +2602,6 @@ consider_iface_release(const struct ovsrec_interface *iface_rec,
>                                             lbinding->iface->name,
>                                             &lbinding->iface->header_.uuid);
>              }
> -
>          } else if (b_lport && b_lport->type == LP_LOCALPORT) {
>              /* lbinding is associated with a localport.  Remove it from the
>               * related lports. */
> @@ -2956,12 +2991,27 @@ handle_updated_vif_lport(const struct sbrec_port_binding *pb,
>  
>  static bool
>  consider_patch_port_for_local_datapaths(const struct sbrec_port_binding *pb,
> +                                        const struct sbrec_port_binding *cr_pb,
>                                          struct binding_ctx_in *b_ctx_in,
> -                                        struct binding_ctx_out *b_ctx_out)
> +                                        struct binding_ctx_out *b_ctx_out,
> +                                        bool check_and_remove_localdps)
>  {
> -    struct local_datapath *ld =
> -        get_local_datapath(b_ctx_out->local_datapaths,
> -                           pb->datapath->tunnel_key);
> +    const struct sbrec_port_binding *peer;
> +    struct local_datapath *peer_ld = NULL;
> +    struct local_datapath *ld = NULL;
> +
> +    ld = get_local_datapath(b_ctx_out->local_datapaths,
> +                            pb->datapath->tunnel_key);
> +    if (ld && ld->has_only_dgp_peer_ports) {
> +        /* Nothing much to do. */
> +        return true;
> +    }
> +
> +    peer = lport_get_peer(pb, b_ctx_in->sbrec_port_binding_by_name);
> +    if (peer) {
> +        peer_ld = get_local_datapath(b_ctx_out->local_datapaths,
> +                                     peer->datapath->tunnel_key);
> +    }
>  
>      if (!ld) {
>          /* If 'ld' for this lport is not present, then check if
> @@ -2969,17 +3019,9 @@ consider_patch_port_for_local_datapaths(const struct sbrec_port_binding *pb,
>           * and peer's datapath is already in the local datapaths,
>           * then add this lport's datapath to the local_datapaths.
>           * */
> -        const struct sbrec_port_binding *peer;
> -        struct local_datapath *peer_ld = NULL;
> -        peer = lport_get_peer(pb, b_ctx_in->sbrec_port_binding_by_name);
> -        if (peer) {
> -            peer_ld =
> -                get_local_datapath(b_ctx_out->local_datapaths,
> -                                   peer->datapath->tunnel_key);
> -        }
> -        if (peer_ld && need_add_peer_to_local(
> -                b_ctx_in->sbrec_port_binding_by_name, peer,
> -                b_ctx_in->chassis_rec)) {
> +        if (peer_ld && !peer_ld->has_only_dgp_peer_ports &&
> +            need_add_peer_to_local(b_ctx_in->sbrec_port_binding_by_name, peer,
> +                                   b_ctx_in->chassis_rec)) {
>              ld = add_local_datapath(
>                      b_ctx_in->sbrec_datapath_binding_by_key,
>                      b_ctx_in->sbrec_port_binding_by_datapath,
> @@ -2992,7 +3034,7 @@ consider_patch_port_for_local_datapaths(const struct sbrec_port_binding *pb,
>          /* Add the peer datapath to the local datapaths if it's
>           * not present yet.
>           */
> -        if (need_add_peer_to_local(
> +        if (peer && need_add_peer_to_local(
>                  b_ctx_in->sbrec_port_binding_by_name, pb,
>                  b_ctx_in->chassis_rec)) {
>              add_local_datapath_peer_port(
> @@ -3003,6 +3045,18 @@ consider_patch_port_for_local_datapaths(const struct sbrec_port_binding *pb,
>                  ld, b_ctx_out->local_datapaths,
>                  b_ctx_out->tracked_dp_bindings);
>          }
> +
> +        if (check_and_remove_localdps) {
> +            bool cleanedup = false;
> +            if (!cleanup_patch_port_local_dps(pb, cr_pb, peer, ld, b_ctx_in,
> +                                              b_ctx_out, &cleanedup)) {
> +                return false;
> +            }
> +
> +            if (cleanedup) {
> +                ld = NULL;
> +            }
> +        }
>      }
>  
>      /* If this chassis is requested - try to claim. */
> @@ -3021,12 +3075,10 @@ consider_patch_port_for_local_datapaths(const struct sbrec_port_binding *pb,
>          || if_status_is_port_claimed(b_ctx_out->if_mgr, pb->logical_port)) {
>  
>          remove_local_lports(pb->logical_port, b_ctx_out);
> -        if (!release_lport(pb, ld, b_ctx_in->chassis_rec,
> -                           !b_ctx_in->ovnsb_idl_txn,
> -                           b_ctx_out->tracked_dp_bindings,
> -                           b_ctx_out->if_mgr)) {
> -            return false;
> -        }
> +        return release_lport(pb, ld, b_ctx_in->chassis_rec,
> +                             !b_ctx_in->ovnsb_idl_txn,
> +                             b_ctx_out->tracked_dp_bindings,
> +                             b_ctx_out->if_mgr);
>      }
>      return true;
>  }
> @@ -3086,8 +3138,8 @@ handle_updated_port(struct binding_ctx_in *b_ctx_in,
>  
>      case LP_PATCH:
>          update_related_lport(pb, b_ctx_out);
> -        handled = consider_patch_port_for_local_datapaths(pb, b_ctx_in,
> -                                                              b_ctx_out);
> +        handled = consider_patch_port_for_local_datapaths(pb, NULL, b_ctx_in,
> +                                                          b_ctx_out, true);
>          break;
>  
>      case LP_VTEP:
> @@ -3130,8 +3182,8 @@ handle_updated_port(struct binding_ctx_in *b_ctx_in,
>              break;
>          }
>          handled = consider_patch_port_for_local_datapaths(distributed_pb,
> -                                                          b_ctx_in,
> -                                                          b_ctx_out);
> +                                                          pb, b_ctx_in,
> +                                                          b_ctx_out, true);
>          break;
>  
>      case LP_EXTERNAL:
> @@ -3901,3 +3953,142 @@ binding_destroy(void)
>      shash_destroy_free_data(&_qos_ports);
>      sset_clear(&_postponed_ports);
>  }
> +
> +static bool
> +is_patch_pb_chassis_relevant(
> +    const struct sbrec_port_binding *pb,
> +    const struct sbrec_chassis *chassis,
> +    struct ovsdb_idl_index *sbrec_port_binding_by_name)
> +{
> +    if (ha_chassis_group_contains(pb->ha_chassis_group, chassis)) {
> +        return true;
> +    }
> +
> +    const struct sbrec_port_binding *pb_crp =
> +        lport_get_cr_port(sbrec_port_binding_by_name, pb);
> +    if (pb_crp) {
> +        return ha_chassis_group_contains(pb_crp->ha_chassis_group, chassis);
> +    }
> +
> +    return false;
> +}
> +
> +static bool
> +cleanup_patch_port_local_dps(const struct sbrec_port_binding *pb,
> +                             const struct sbrec_port_binding *cr_pb,
> +                             const struct sbrec_port_binding *peer,
> +                             struct local_datapath *ld,
> +                             struct binding_ctx_in *b_ctx_in,
> +                             struct binding_ctx_out *b_ctx_out,
> +                             bool *cleanedup)
> +{
> +    *cleanedup = false;
> +    if (!peer) {
> +        /* Remove 'pb' from the ld's peer ports as it has no peer. */
> +        remove_local_datapath_peer_port(pb, ld,
> +                                        b_ctx_out->local_datapaths);
> +    }
> +
> +    /* We can consider removing the 'ld' of the patch port 'pb' from the
> +     * local datapaths, if all the below conditions are met
> +     *     - 'pb' doesn't have a peer or ld' is a router datapath
> +     *     - if 'pb' is a distributed gateway port (dgp), then
> +     *       its chassisredirect port's ha chassis group doesn't
> +     *       contain our 'chassis rec'
> +     *     - and finally 'ld' is not relevant any more.  See
> +     *       local_datapath_is_relevant() for more details.
> +     *
> +     * Note: If 'ld' can be removed, then all its connected local datapaths
> +     * can also be removed.
> +     *
> +     * For example, if we had sw1-port1 ->  sw1 -> lr1 -> sw2 and if
> +     * sw1-port1 resides on this chassis, and if the link between sw1 and
> +     * lr1 is broken, then we can remove lr1 and sw2 from the
> +     * local_datapaths.
> +     * */
> +
> +    bool consider_ld_for_removal = !peer || !ld->is_switch;
> +    if (consider_ld_for_removal && cr_pb) {
> +        consider_ld_for_removal = !ha_chassis_group_contains(
> +            cr_pb->ha_chassis_group, b_ctx_in->chassis_rec);
> +    }
> +
> +    if (!consider_ld_for_removal) {
> +        return true;
> +    }
> +
> +    int depth = 0;
> +
> +    bool is_relevant = local_datapath_is_relevant(
> +        ld, NULL, b_ctx_out->local_datapaths,
> +        &depth, b_ctx_in->chassis_rec,
> +        b_ctx_in->sbrec_port_binding_by_name);
> +
> +    if (depth >= 100) {
> +        /* datapaths are too deeply nested.  Fall back to recompute. */
> +        return false;
> +    }
> +
> +    if (!is_relevant) {
> +        /* This 'ld' can be removed from the local datapaths as
> +            *   - its a router datapath and
> +            *   - it has no peers locally. */
> +        local_datapath_remove_and_destroy(ld, b_ctx_out->local_datapaths,
> +                                          b_ctx_out->tracked_dp_bindings);
> +        *cleanedup = true;
> +    }
> +
> +    return true;
> +}
> +
> +static bool
> +local_datapath_is_relevant(struct local_datapath *ld,
> +                           struct local_datapath *ignore_peer_ld,
> +                           struct hmap *local_datapaths, int *depth,
> +                           const struct sbrec_chassis *chassis,
> +                           struct ovsdb_idl_index *sbrec_pb_by_name)
> +{
> +    if (!sset_is_empty(&ld->claimed_lports) ||
> +        !shash_is_empty(&ld->external_ports) ||
> +        !shash_is_empty(&ld->multichassis_ports) ||
> +        ld->vtep_port) {
> +        return true;
> +    }
> +
> +    bool relevant = false;
> +
> +    if (*depth >= 100) {
> +        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> +        VLOG_WARN_RL(&rl, "datapaths nested too deep");
> +        return true;
> +    }
> +
> +    for (size_t i = 0; i < ld->n_peer_ports && !relevant; i++) {
> +        const struct sbrec_port_binding *remote = ld->peer_ports[i].remote;
> +        const struct sbrec_port_binding *local = ld->peer_ports[i].local;
> +
> +        if (is_patch_pb_chassis_relevant(local, chassis,
> +                                         sbrec_pb_by_name)) {
> +            return  true;
> +        }
> +
> +        if (is_patch_pb_chassis_relevant(remote, chassis,
> +                                         sbrec_pb_by_name)) {
> +            return  true;
> +        }
> +
> +        struct local_datapath *peer_ld;
> +        uint32_t remote_peer_ld_key;
> +        remote_peer_ld_key = ld->peer_ports[i].remote->datapath->tunnel_key;
> +        peer_ld = get_local_datapath(local_datapaths, remote_peer_ld_key);
> +        if (peer_ld && !peer_ld->has_only_dgp_peer_ports &&
> +            peer_ld != ignore_peer_ld) {
> +            *depth = *depth + 1;
> +            relevant = local_datapath_is_relevant(peer_ld, ld,
> +                                                  local_datapaths, depth,
> +                                                  chassis, sbrec_pb_by_name);
> +        }
> +    }
> +
> +    return relevant;
> +}
> diff --git a/controller/binding.h b/controller/binding.h
> index d13ae36c79..a4346c3e10 100644
> --- a/controller/binding.h
> +++ b/controller/binding.h
> @@ -201,6 +201,8 @@ bool binding_handle_port_binding_changes(struct binding_ctx_in *,
>  void binding_tracked_dp_destroy(struct hmap *tracked_datapaths);
>  
>  void binding_dump_local_bindings(struct local_binding_data *, struct ds *);
> +void binding_dump_local_datapaths(struct hmap *local_datapaths,
> +                                  struct ds *out_data);
>  
>  void binding_dump_related_lports(struct related_lports *related_lports,
>                                   struct ds *);
> diff --git a/controller/local_data.c b/controller/local_data.c
> index e19b2bf865..bb5740f1d6 100644
> --- a/controller/local_data.c
> +++ b/controller/local_data.c
> @@ -53,6 +53,13 @@ static struct tracked_datapath *tracked_datapath_create(
>  
>  static bool datapath_is_switch(const struct sbrec_datapath_binding *);
>  static bool datapath_is_transit_switch(const struct sbrec_datapath_binding *);
> +static bool datapath_has_only_dgp_peer_ports(
> +    const struct sbrec_datapath_binding *);
> +static void local_datapath_remove_and_destroy__(
> +    struct local_datapath *ld,
> +    const struct sbrec_port_binding *ignore_peer_port,
> +    struct hmap *local_datapaths,
> +    struct hmap *tracked_datapaths);
>  
>  static uint64_t local_datapath_usage;
>  
> @@ -86,6 +93,7 @@ local_datapath_alloc(const struct sbrec_datapath_binding *dp)
>      ld->datapath = dp;
>      ld->is_switch = datapath_is_switch(dp);
>      ld->is_transit_switch = datapath_is_transit_switch(dp);
> +    ld->has_only_dgp_peer_ports = datapath_has_only_dgp_peer_ports(dp);
>      shash_init(&ld->external_ports);
>      shash_init(&ld->multichassis_ports);
>      sset_init(&ld->claimed_lports);
> @@ -132,6 +140,14 @@ local_datapath_destroy(struct local_datapath *ld)
>      free(ld);
>  }
>  
> +void local_datapath_remove_and_destroy(struct local_datapath *ld,
> +                                       struct hmap *local_datapaths,
> +                                       struct hmap *tracked_datapaths)
> +{
> +    local_datapath_remove_and_destroy__(ld, NULL, local_datapaths,
> +                                        tracked_datapaths);
> +}
> +
>  /* Checks if pb is running on local gw router or pb is a patch port
>   * and the peer datapath should be added to local datapaths. */
>  bool
> @@ -226,12 +242,12 @@ add_local_datapath_peer_port(
>          get_local_datapath(local_datapaths,
>                             peer->datapath->tunnel_key);
>      if (!peer_ld) {
> -        add_local_datapath__(sbrec_datapath_binding_by_key,
> -                             sbrec_port_binding_by_datapath,
> -                             sbrec_port_binding_by_name, 1,
> -                             peer->datapath, chassis, local_datapaths,
> -                             tracked_datapaths);
> -        return;
> +        peer_ld = add_local_datapath__(sbrec_datapath_binding_by_key,
> +                                       sbrec_port_binding_by_datapath,
> +                                       sbrec_port_binding_by_name, 1,
> +                                       peer->datapath, chassis,
> +                                       local_datapaths,
> +                                       tracked_datapaths);
>      }
>  
>      local_datapath_peer_port_add(peer_ld, peer, pb);
> @@ -613,6 +629,17 @@ add_local_datapath__(struct ovsdb_idl_index *sbrec_datapath_binding_by_key,
>                               tracked_datapaths);
>      }
>  
> +    if (ld->has_only_dgp_peer_ports) {
> +        /* If this flag is set, it means this 'switch' datapath has
> +         *  - one ore many localnet ports.
> +         *  - all the router ports it is connected to are
> +         *    distributed gateway ports (DGPs).
> +         * There is no need to add the routers of the dgps to
> +         * the local datapaths.
> +         * */
> +        return ld;
> +    }
> +
>      if (depth >= 100) {
>          static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
>          VLOG_WARN_RL(&rl, "datapaths nested too deep");
> @@ -705,6 +732,13 @@ datapath_is_transit_switch(const struct sbrec_datapath_binding *ldp)
>      return smap_get(&ldp->external_ids, "interconn-ts") != NULL;
>  }
>  
> +static bool
> +datapath_has_only_dgp_peer_ports(const struct sbrec_datapath_binding *ldp)
> +{
> +    return datapath_is_switch(ldp) &&
> +           smap_get_bool(&ldp->external_ids, "only_dgp_peer_ports", false);
> +}
> +
>  bool
>  lb_is_local(const struct sbrec_load_balancer *sbrec_lb,
>              const struct hmap *local_datapaths)
> @@ -745,3 +779,41 @@ lb_is_local(const struct sbrec_load_balancer *sbrec_lb,
>  
>      return false;
>  }
> +
> +static void
> +local_datapath_remove_and_destroy__(struct local_datapath *ld,
> +                                    const struct sbrec_port_binding *ignore_pb,
> +                                    struct hmap *local_datapaths,
> +                                    struct hmap *tracked_datapaths)
> +{
> +    for (size_t i = 0; i < ld->n_peer_ports; i++) {
> +        const struct sbrec_port_binding *remote = ld->peer_ports[i].remote;
> +        const struct sbrec_port_binding *local = ld->peer_ports[i].local;
> +
> +        if (local == ignore_pb) {
> +            continue;
> +        }
> +
> +        struct local_datapath *peer_ld;
> +        uint32_t remote_peer_ld_key;
> +
> +        remote_peer_ld_key = ld->peer_ports[i].remote->datapath->tunnel_key;
> +        peer_ld = get_local_datapath(local_datapaths, remote_peer_ld_key);
> +        if (peer_ld && !peer_ld->has_only_dgp_peer_ports) {
> +            local_datapath_remove_and_destroy__(peer_ld, remote,
> +                                                local_datapaths,
> +                                                tracked_datapaths);
> +        } else if (peer_ld && peer_ld->has_only_dgp_peer_ports) {
> +            remove_local_datapath_peer_port(ld->peer_ports[i].remote,
> +                                            peer_ld, local_datapaths);
> +        }
> +    }
> +
> +    hmap_remove(local_datapaths, &ld->hmap_node);
> +    if (tracked_datapaths) {
> +        tracked_datapath_add(ld->datapath, TRACKED_RESOURCE_REMOVED,
> +                             tracked_datapaths);
> +    }
> +
> +    local_datapath_destroy(ld);
> +}
> diff --git a/controller/local_data.h b/controller/local_data.h
> index 857d63a51d..ef01f4259e 100644
> --- a/controller/local_data.h
> +++ b/controller/local_data.h
> @@ -46,6 +46,8 @@ struct local_datapath {
>      const struct sbrec_datapath_binding *datapath;
>      bool is_switch;
>      bool is_transit_switch;
> +    /* Valid only for 'is_switch' local datapath. */
> +    bool has_only_dgp_peer_ports;
>  
>      /* The localnet port in this datapath, if any (at most one is allowed). */
>      const struct sbrec_port_binding *localnet_port;
> @@ -91,6 +93,10 @@ struct local_datapath * add_local_datapath(
>  
>  void local_datapaths_destroy(struct hmap *local_datapaths);
>  void local_datapath_destroy(struct local_datapath *ld);
> +void local_datapath_remove_and_destroy(struct local_datapath *,
> +                                       struct hmap *local_datapaths,
> +                                       struct hmap *tracked_datapaths);
> +
>  void add_local_datapath_peer_port(
>      const struct sbrec_port_binding *,
>      const struct sbrec_chassis *,
> diff --git a/controller/lport.c b/controller/lport.c
> index 8bc230e896..f164803528 100644
> --- a/controller/lport.c
> +++ b/controller/lport.c
> @@ -102,6 +102,18 @@ lport_get_l3gw_peer(const struct sbrec_port_binding *pb,
>      return get_peer_lport(pb, sbrec_port_binding_by_name);
>  }
>  
> +const struct sbrec_port_binding *
> +lport_get_cr_port(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> +                  const struct sbrec_port_binding *pb)
> +{
> +    const char *crp = smap_get(&pb->options, "chassis-redirect-port");
> +    if (crp) {
> +        return lport_lookup_by_name(sbrec_port_binding_by_name, crp);
> +    }
> +
> +    return NULL;
> +}
> +
>  enum can_bind
>  lport_can_bind_on_this_chassis(const struct sbrec_chassis *chassis_rec,
>                                 const struct sbrec_port_binding *pb)
> diff --git a/controller/lport.h b/controller/lport.h
> index 3a58ebe39e..069bd59096 100644
> --- a/controller/lport.h
> +++ b/controller/lport.h
> @@ -73,4 +73,8 @@ const struct sbrec_port_binding *lport_get_l3gw_peer(
>  bool
>  lport_is_activated_by_activation_strategy(const struct sbrec_port_binding *pb,
>                                            const struct sbrec_chassis *chassis);
> +const struct sbrec_port_binding *lport_get_cr_port(
> +    struct ovsdb_idl_index *sbrec_port_binding_by_name,
> +    const struct sbrec_port_binding *);
> +
>  #endif /* controller/lport.h */
> diff --git a/controller/ovn-controller.c b/controller/ovn-controller.c
> index da942abaa9..dc8ceec076 100644
> --- a/controller/ovn-controller.c
> +++ b/controller/ovn-controller.c
> @@ -101,6 +101,7 @@ static unixctl_cb_func debug_pause_execution;
>  static unixctl_cb_func debug_resume_execution;
>  static unixctl_cb_func debug_status_execution;
>  static unixctl_cb_func debug_dump_local_bindings;
> +static unixctl_cb_func debug_dump_local_datapaths;
>  static unixctl_cb_func debug_dump_related_lports;
>  static unixctl_cb_func debug_dump_local_template_vars;
>  static unixctl_cb_func debug_dump_local_mac_bindings;
> @@ -1688,6 +1689,22 @@ runtime_data_sb_datapath_binding_handler(struct engine_node *node OVS_UNUSED,
>                  return false;
>              }
>          }
> +
> +        if (sbrec_datapath_binding_is_updated(
> +                dp, SBREC_DATAPATH_BINDING_COL_EXTERNAL_IDS) &&
> +            !sbrec_datapath_binding_is_new(dp)) {
> +            struct local_datapath *ld =
> +                get_local_datapath(&rt_data->local_datapaths,
> +                                   dp->tunnel_key);
> +                if (ld && ld->is_switch) {
> +                    bool only_dgp_peer_ports =
> +                        smap_get_bool(&dp->external_ids, "only_dgp_peer_ports",
> +                                      false);
> +                    if (ld->has_only_dgp_peer_ports != only_dgp_peer_ports) {
> +                        return false;
> +                    }
> +                }
> +        }
>      }
>  
>      return true;
> @@ -4357,6 +4374,12 @@ lflow_output_runtime_data_handler(struct engine_node *node,
>      init_lflow_ctx(node, fo, &l_ctx_in, &l_ctx_out);
>  
>      struct tracked_datapath *tdp;
> +    HMAP_FOR_EACH (tdp, node, tracked_dp_bindings) {
> +        if (tdp->tracked_type == TRACKED_RESOURCE_REMOVED) {
> +            return false;
> +        }
> +    }
> +
>      HMAP_FOR_EACH (tdp, node, tracked_dp_bindings) {
>          if (tdp->tracked_type == TRACKED_RESOURCE_NEW) {
>              if (!lflow_add_flows_for_datapath(tdp->dp, &l_ctx_in,
> @@ -5552,6 +5575,10 @@ main(int argc, char *argv[])
>                               debug_dump_local_bindings,
>                               &runtime_data->lbinding_data);
>  
> +    unixctl_command_register("debug/dump-local-datapaths", "", 0, 0,
> +                             debug_dump_local_datapaths,
> +                             &runtime_data->local_datapaths);
> +
>      unixctl_command_register("debug/dump-related-ports", "", 0, 0,
>                               debug_dump_related_lports,
>                               &runtime_data->related_lports);
> @@ -6517,6 +6544,17 @@ debug_dump_local_bindings(struct unixctl_conn *conn, int argc OVS_UNUSED,
>      ds_destroy(&binding_data);
>  }
>  
> +static void
> +debug_dump_local_datapaths(struct unixctl_conn *conn, int argc OVS_UNUSED,
> +                           const char *argv[] OVS_UNUSED,
> +                           void *local_datapaths)
> +{
> +    struct ds local_dps_data = DS_EMPTY_INITIALIZER;
> +    binding_dump_local_datapaths(local_datapaths, &local_dps_data);
> +    unixctl_command_reply(conn, ds_cstr(&local_dps_data));
> +    ds_destroy(&local_dps_data);
> +}
> +
>  static void
>  debug_dump_related_lports(struct unixctl_conn *conn, int argc OVS_UNUSED,
>                            const char *argv[] OVS_UNUSED, void *related_lports)
> diff --git a/tests/multinode.at b/tests/multinode.at
> index c1bd3123ac..b828f16135 100644
> --- a/tests/multinode.at
> +++ b/tests/multinode.at
> @@ -2776,3 +2776,181 @@ for i in 1 2; do
>  done
>  
>  AT_CLEANUP
> +
> +AT_SETUP([ovn multinode - only_dgp_peer_ports provider switch functionality])
> +
> +# Check that ovn-fake-multinode setup is up and running
> +check_fake_multinode_setup
> +
> +# Delete the multinode NB and OVS resources before starting the test.
> +cleanup_multinode_resources
> +
> +check multinode_nbctl ls-add sw0
> +check multinode_nbctl lsp-add sw0 sw0-port1
> +check multinode_nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03 10.0.0.3 1000::3"
> +check multinode_nbctl lsp-add sw0 sw0-port2
> +check multinode_nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:04 10.0.0.4 1000::4"
> +
> +check multinode_nbctl lr-add lr0
> +check multinode_nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 1000::1/64
> +check multinode_nbctl lsp-add sw0 sw0-lr0
> +check multinode_nbctl lsp-set-type sw0-lr0 router
> +check multinode_nbctl lsp-set-addresses sw0-lr0 router
> +check multinode_nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> +
> +check multinode_nbctl ls-add public
> +check multinode_nbctl lrp-add lr0 lr0-public 00:00:20:20:12:13 172.16.1.100/24 2000::1/64
> +check multinode_nbctl lsp-add public public-lr0
> +check multinode_nbctl lsp-set-type public-lr0 router
> +check multinode_nbctl lsp-set-addresses public-lr0 router
> +check multinode_nbctl lsp-set-options public-lr0 router-port=lr0-public
> +
> +# localnet port
> +check multinode_nbctl lsp-add public ln-public
> +check multinode_nbctl lsp-set-type ln-public localnet
> +check multinode_nbctl lsp-set-addresses ln-public unknown
> +check multinode_nbctl lsp-set-options ln-public network_name=public
> +
> +check multinode_nbctl lrp-set-gateway-chassis lr0-public ovn-gw-1 20
> +check multinode_nbctl lr-nat-add lr0 snat 172.16.1.100 10.0.0.0/24
> +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.16.1.110 10.0.0.3 sw0-port1 50:54:00:00:00:03
> +check multinode_nbctl lr-nat-add lr0 snat 2000::1 1000::/64
> +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 2000::2 1000::3 sw0-port1 50:54:00:00:00:03
> +
> +check multinode_nbctl --wait=hv sync
> +
> +check multinode_nbctl ls-add sw1
> +check multinode_nbctl lsp-add sw1 sw1-port1
> +check multinode_nbctl lsp-set-addresses sw1-port1 "40:54:00:00:00:03 20.0.0.3 2000::3"
> +
> +check multinode_nbctl lr-add lr1
> +check multinode_nbctl lrp-add lr1 lr1-sw1 00:00:01:00:ef:01 20.0.0.1/24 2000::a/64
> +check multinode_nbctl lsp-add sw1 sw1-lr1
> +check multinode_nbctl lsp-set-type sw1-lr1 router
> +check multinode_nbctl lsp-set-addresses sw1-lr1 router
> +check multinode_nbctl lsp-set-options sw1-lr1 router-port=lr1-sw1
> +
> +check multinode_nbctl lrp-add lr1 lr1-public 00:00:20:30:22:13 172.16.1.101/24
> +check multinode_nbctl lsp-add public public-lr1
> +check multinode_nbctl lsp-set-type public-lr1 router
> +check multinode_nbctl lsp-set-addresses public-lr1 router
> +check multinode_nbctl lsp-set-options public-lr1 router-port=lr1-public
> +
> +check multinode_nbctl lr-nat-add lr1 snat 172.16.1.101 20.0.0.0/24
> +check multinode_nbctl lr-nat-add lr1 dnat_and_snat 172.16.1.120 20.0.0.3
> +check multinode_nbctl lrp-set-gateway-chassis lr1-public ovn-gw-1 20
> +
> +check multinode_nbctl --wait=hv sync
> +
> +m_as ovn-chassis-1 /data/create_fake_vm.sh sw0-port1 sw0p1 50:54:00:00:00:03 1342 10.0.0.3 24 10.0.0.1 1000::3/64 1000::a
> +m_as ovn-chassis-2 /data/create_fake_vm.sh sw1-port1 sw1p1 40:54:00:00:00:03 1342 20.0.0.3 24 20.0.0.1 2000::4/64 1000::a
> +
> +m_wait_for_ports_up sw0-port1
> +m_wait_for_ports_up sw1-port1
> +
> +m_as ovn-central-az1-1 ovn-sbctl show
> +
> +m_as ovn-chassis-1 ovn-appctl  debug/dump-local-datapaths | sort
> +m_as ovn-chassis-2 ovn-appctl  debug/dump-local-datapaths | sort
> +
> +AT_CHECK([m_as ovn-chassis-1 ovn-appctl  debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([m_as ovn-chassis-2 ovn-appctl  debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: sw1, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([m_as ovn-gw-1 ovn-appctl  debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: lr1, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Datapath: sw1, type: switch
> +Local datapaths:
> +])
> +
> +# ping lr0-public IP - 172.168.0.100
> +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.16.1.100 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +# ping lr1-public IP - 172.168.0.101 from sw0p1
> +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.16.1.101 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +# ping public ip of sw1-port1 - 172.16.1.120 from sw0p1
> +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.16.1.120 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +# ping public ip of sw0-port1 - 172.16.1.110 from sw1p1
> +M_NS_CHECK_EXEC([ovn-chassis-2], [sw1p1], [ping -q -c 3 -i 0.3 -w 2 172.16.1.110 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +# Bind sw0-port2 on chassis-2
> +m_as ovn-chassis-2 /data/create_fake_vm.sh sw0-port2 sw0p2 50:54:00:00:00:04 1342 10.0.0.4 24 10.0.0.1 1000::4/64 1000::a
> +m_wait_for_ports_up sw0-port2
> +
> +AT_CHECK([m_as ovn-chassis-2 ovn-appctl  debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: lr1, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Datapath: sw1, type: switch
> +Local datapaths:
> +])
> +
> +# ping public ip of sw0-port1 - 172.16.1.110 from sw0p2
> +M_NS_CHECK_EXEC([ovn-chassis-2], [sw0p2], [ping -q -c 3 -i 0.3 -w 2 172.16.1.110 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +# ping public ip of sw1-port1 - 172.16.1.120 from sw0p2
> +M_NS_CHECK_EXEC([ovn-chassis-2], [sw0p2], [ping -q -c 3 -i 0.3 -w 2 172.16.1.120 | FORMAT_PING], \
> +[0], [dnl
> +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> +])
> +
> +# Create a normal router port in public with its peer as a normal distributed router port.
> +check multinode_nbctl lsp-add public public-lr2
> +check multinode_nbctl lsp-set-type public-lr2 router
> +check multinode_nbctl lsp-set-addresses public-lr2 router
> +check multinode_nbctl lsp-set-options public-lr2 router-port=lr2-public
> +check multinode_nbctl lr-add lr2
> +check multinode_nbctl lrp-add lr2 lr2-public 00:00:41:00:1f:61 172.16.1.102/24 3000::a/64
> +
> +check multinode_nbctl --wait=hv sync
> +AT_CHECK([m_as ovn-chassis-1 ovn-appctl  debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Datapath: sw1, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([m_as ovn-chassis-2 ovn-appctl  debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Datapath: sw1, type: switch
> +Local datapaths:
> +])
> +
> +AT_CLEANUP
> diff --git a/tests/ovn-performance.at b/tests/ovn-performance.at
> index 7d480c20c8..a003fc36bc 100644
> --- a/tests/ovn-performance.at
> +++ b/tests/ovn-performance.at
> @@ -479,7 +479,7 @@ OVN_CONTROLLER_EXPECT_NO_HIT(
>  )
>  
>  OVN_CONTROLLER_EXPECT_HIT_COND(
> -    [hv1 hv2 hv3 hv4 hv5], [lflow_run], [=0 =0 >0 =0 =0],
> +    [hv1 hv2 hv3 hv4 hv5], [lflow_run], [>0 >0 >0 =0 =0],
>      [ovn-nbctl --wait=hv lrp-set-gateway-chassis lr1-public hv3 30 && ovn-nbctl --wait=hv sync]
>  )
>  
> @@ -552,8 +552,8 @@ hv5_ch=$(ovn-sbctl --bare --columns _uuid list chassis hv5)
>  OVS_WAIT_UNTIL([ovn-sbctl find port_binding logical_port=cr-lr1-public chassis=$hv5_ch])
>  check ovn-nbctl --wait=hv sync
>  # Delete hv5 from gateway chassis. There should be no lflow_run.
> -OVN_CONTROLLER_EXPECT_NO_HIT(
> -    [hv1 hv2 hv3 hv4 hv5], [lflow_run],
> +OVN_CONTROLLER_EXPECT_HIT_COND(
> +    [hv1 hv2 hv3 hv4 hv5], [lflow_run], [=0 =0 =0 =0 =0]
>      [ovn-nbctl --wait=hv lrp-del-gateway-chassis lr1-public hv5]
>  )
>  
> diff --git a/tests/ovn.at b/tests/ovn.at
> index d105ed2535..d4e87212a5 100644
> --- a/tests/ovn.at
> +++ b/tests/ovn.at
> @@ -41481,6 +41481,859 @@ OVN_CLEANUP([hv1])
>  AT_CLEANUP
>  ])
>  
> +OVN_FOR_EACH_NORTHD_NO_HV([
> +AT_SETUP([ovn-controller -- only_dgp_peer_ports flag in SB datapath_binding])
> +AT_KEYWORDS([multiple-l3dgw-ports])
> +ovn_start
> +net_add n1
> +sim_add hv1
> +as hv1
> +check ovs-vsctl add-br br-phys
> +ovn_attach n1 br-phys 192.168.0.1
> +
> +sim_add hv2
> +as hv2
> +check ovs-vsctl add-br br-phys
> +ovn_attach n1 br-phys 192.168.0.2
> +
> +sim_add gw1
> +as gw1
> +check ovs-vsctl add-br br-phys
> +ovn_attach n1 br-phys 192.168.0.3
> +
> +sim_add gw2
> +as gw2
> +check ovs-vsctl add-br br-phys
> +ovn_attach n1 br-phys 192.168.0.4
> +
> +check ovn-nbctl ls-add sw0
> +check ovn-nbctl lsp-add sw0 sw0-port1
> +check ovn-nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:01 10.0.0.3 1000::3"
> +check ovn-nbctl lsp-add sw0 sw0-port2
> +check ovn-nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:02 10.0.0.4 1000::4"
> +
> +check ovn-nbctl lr-add lr0
> +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 1000::1/64
> +check ovn-nbctl lsp-add sw0 sw0-lr0
> +check ovn-nbctl lsp-set-type sw0-lr0 router
> +check ovn-nbctl lsp-set-addresses sw0-lr0 router
> +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> +
> +check ovn-nbctl ls-add public
> +check ovn-nbctl lrp-add lr0 lr0-public 00:00:20:20:12:13 172.168.0.100/24 2000::1/64
> +check ovn-nbctl lsp-add public public-lr0
> +check ovn-nbctl lsp-set-type public-lr0 router
> +check ovn-nbctl lsp-set-addresses public-lr0 router
> +check ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public
> +
> +# localnet port
> +check ovn-nbctl lsp-add public ln-public
> +check ovn-nbctl lsp-set-type ln-public localnet
> +check ovn-nbctl lsp-set-addresses ln-public unknown
> +check ovn-nbctl lsp-set-options ln-public network_name=phys
> +
> +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1 20
> +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 10.0.0.0/24
> +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 sw0-port2 f0:00:00:01:02:04
> +check ovn-nbctl lr-nat-add lr0 snat 2000::1 1000::/64
> +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 2000::2 1000::4 sw0-port2 f0:00:00:01:02:04
> +
> +check ovn-nbctl --wait=hv sync
> +
> +sw0_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=sw0))
> +lr0_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=lr0))
> +public_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=public))
> +
> +check_offlows_for_datapath() {
> +    hv=$1
> +    dp_key=$2
> +    should_be_present=$3
> +
> +    if [[ "$should_be_present" == "yes" ]]; then
> +        echo "Flows should be present for hv - $hv : datapath - $dp_key"
> +        OVS_WAIT_UNTIL(
> +            [test $(as $hv ovs-ofctl dump-flows br-int | grep -c metadata=0x$dp_key) -gt 0]
> +        )
> +    else
> +        echo "Flows should NOT be present for hv - $hv : datapath - $dp_key"
> +        OVS_WAIT_UNTIL(
> +            [test $(as $hv ovs-ofctl dump-flows br-int | grep -c metadata=0x$dp_key) -eq 0]
> +        )
> +    fi
> +}
> +
> +AT_CHECK([ovn-sbctl get datapath_binding public external_ids:only_dgp_peer_ports], [0], [dnl
> +"true"
> +])
> +
> +check_offlows_for_datapath hv1 $sw0_dp_key no
> +check_offlows_for_datapath hv1 $lr0_dp_key no
> +check_offlows_for_datapath hv1 $public_dp_key no
> +
> +check_offlows_for_datapath hv2 $sw0_dp_key no
> +check_offlows_for_datapath hv2 $lr0_dp_key no
> +check_offlows_for_datapath hv2 $public_dp_key no
> +
> +check_offlows_for_datapath gw1 $sw0_dp_key yes
> +check_offlows_for_datapath gw1 $lr0_dp_key yes
> +check_offlows_for_datapath gw1 $public_dp_key yes
> +
> +check_offlows_for_datapath gw2 $sw0_dp_key no
> +check_offlows_for_datapath gw2 $lr0_dp_key no
> +check_offlows_for_datapath gw2 $public_dp_key no
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> +Local datapaths:
> +])
> +
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> +Local datapaths:
> +])
> +
> +# Create a VIF on hv1 for sw0-port1
> +AS_BOX([create a VIF on hv1 for sw0-port1])
> +
> +as hv1
> +ovs-vsctl -- add-port br-int hv1-vif1 -- \
> +    set interface hv1-vif1 external-ids:iface-id=sw0-port1 \
> +    options:tx_pcap=hv1/vif1-tx.pcap \
> +    options:rxq_pcap=hv1/vif1-rx.pcap \
> +    ofport-request=1
> +
> +wait_for_ports_up sw0-port1
> +
> +AS_BOX([Create a VIF on hv1 for sw0-port1 - hv1 should have flows for sw0, lr0 and public])
> +
> +check_offlows_for_datapath hv1 $sw0_dp_key yes
> +check_offlows_for_datapath hv1 $lr0_dp_key yes
> +check_offlows_for_datapath hv1 $public_dp_key yes
> +
> +AS_BOX([hv2 should NOT have flows for sw0, lr0 and public])
> +check_offlows_for_datapath hv2 $sw0_dp_key no
> +check_offlows_for_datapath hv2 $lr0_dp_key no
> +check_offlows_for_datapath hv2 $public_dp_key no
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> +Local datapaths:
> +])
> +
> +AS_BOX([create a switch sw1 and router lr1, attach both and attach lr1 to public])
> +
> +check ovn-nbctl ls-add sw1
> +check ovn-nbctl lsp-add sw1 sw1-port1
> +check ovn-nbctl lsp-set-addresses sw1-port1 "60:54:00:00:00:01 20.0.0.3"
> +
> +check ovn-nbctl lr-add lr1
> +check ovn-nbctl lrp-add lr1 lr1-sw1 00:00:01:00:ef:01 20.0.0.1/24
> +check ovn-nbctl lsp-add sw1 sw1-lr1
> +check ovn-nbctl lsp-set-type sw1-lr1 router
> +check ovn-nbctl lsp-set-addresses sw1-lr1 router
> +check ovn-nbctl lsp-set-options sw1-lr1 router-port=lr1-sw1
> +
> +check ovn-nbctl lrp-add lr1 lr1-public 00:00:20:30:22:13 172.168.0.101/24
> +check ovn-nbctl lsp-add public public-lr1
> +check ovn-nbctl lsp-set-type public-lr1 router
> +check ovn-nbctl lsp-set-addresses public-lr1 router
> +check ovn-nbctl lsp-set-options public-lr1 router-port=lr1-public
> +
> +sw1_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=sw1))
> +lr1_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=lr1))
> +
> +check ovn-nbctl lr-nat-add lr1 snat 172.168.0.101 20.0.0.0/24
> +check ovn-nbctl lr-nat-add lr1 dnat_and_snat 172.168.0.140 20.0.0.3
> +
> +AS_BOX([create a switch sw2 and router lr2, attach both and attach lr2 to public])
> +
> +check ovn-nbctl ls-add sw2
> +check ovn-nbctl lsp-add sw2 sw2-port1
> +check ovn-nbctl lsp-set-addresses sw2-port1 "70:54:00:00:00:01 30.0.0.3"
> +
> +check ovn-nbctl lr-add lr2
> +check ovn-nbctl lrp-add lr2 lr2-sw2 00:00:02:00:ef:01 30.0.0.1/24
> +check ovn-nbctl lsp-add sw2 sw2-lr2
> +check ovn-nbctl lsp-set-type sw2-lr2 router
> +check ovn-nbctl lsp-set-addresses sw2-lr2 router
> +check ovn-nbctl lsp-set-options sw2-lr2 router-port=lr2-sw2
> +
> +check ovn-nbctl lrp-add lr2 lr2-public 00:00:20:40:22:53 172.168.0.102/24
> +check ovn-nbctl lsp-add public public-lr2
> +check ovn-nbctl lsp-set-type public-lr2 router
> +check ovn-nbctl lsp-set-addresses public-lr2 router
> +check ovn-nbctl lsp-set-options public-lr2 router-port=lr2-public
> +
> +sw2_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=sw2))
> +lr2_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=lr2))
> +
> +check ovn-nbctl lr-nat-add lr2 snat 172.168.0.102 30.0.0.0/24
> +check ovn-nbctl lr-nat-add lr2 dnat_and_snat 172.168.0.150 30.0.0.3
> +
> +check ovn-nbctl --wait=hv sync
> +
> +# Since lr1-public is not a DGP,  public is not a "only_dgp_peer_ports".
> +AT_CHECK([ovn-sbctl get datapath_binding public external_ids:only_dgp_peer_ports], [1], [ignore], [ignore])
> +
> +check_offlows_for_datapath hv1 $sw1_dp_key yes
> +check_offlows_for_datapath hv1 $lr1_dp_key yes
> +check_offlows_for_datapath hv1 $sw2_dp_key yes
> +check_offlows_for_datapath hv1 $lr2_dp_key yes
> +
> +check_offlows_for_datapath hv2 $sw1_dp_key no
> +check_offlows_for_datapath hv2 $lr1_dp_key no
> +check_offlows_for_datapath hv2 $sw2_dp_key no
> +check_offlows_for_datapath hv2 $lr2_dp_key no
> +
> +check_offlows_for_datapath gw1 $sw1_dp_key yes
> +check_offlows_for_datapath gw1 $lr1_dp_key yes
> +check_offlows_for_datapath gw1 $sw2_dp_key yes
> +check_offlows_for_datapath gw1 $lr2_dp_key yes
> +
> +check_offlows_for_datapath gw2 $sw1_dp_key no
> +check_offlows_for_datapath gw2 $lr1_dp_key no
> +check_offlows_for_datapath gw2 $sw2_dp_key no
> +check_offlows_for_datapath gw2 $lr2_dp_key no
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Datapath: sw1, type: switch
> +Datapath: sw2, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Datapath: sw1, type: switch
> +Datapath: sw2, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> +Local datapaths:
> +])
> +
> +AS_BOX([Set gw2 as gateway chassis for lr1-public and lr2-public])
> +check ovn-nbctl --wait=hv lrp-set-gateway-chassis lr1-public gw2 20
> +check ovn-nbctl --wait=hv lrp-set-gateway-chassis lr2-public gw2 30
> +wait_row_count Port_Binding 1 logical_port=cr-lr1-public
> +wait_row_count Port_Binding 1 logical_port=cr-lr2-public
> +
> +AT_CHECK([ovn-sbctl get datapath_binding public external_ids:only_dgp_peer_ports], [0], [dnl
> +"true"
> +])
> +
> +check ovn-nbctl --wait=hv sync
> +
> +check_offlows_for_datapath hv1 $sw0_dp_key yes
> +check_offlows_for_datapath hv1 $lr0_dp_key yes
> +check_offlows_for_datapath hv1 $public_dp_key yes
> +check_offlows_for_datapath hv1 $sw1_dp_key no
> +check_offlows_for_datapath hv1 $lr1_dp_key no
> +check_offlows_for_datapath hv1 $sw2_dp_key no
> +check_offlows_for_datapath hv1 $lr2_dp_key no
> +
> +check_offlows_for_datapath hv2 $sw0_dp_key no
> +check_offlows_for_datapath hv2 $lr0_dp_key no
> +check_offlows_for_datapath hv2 $public_dp_key no
> +check_offlows_for_datapath hv2 $sw1_dp_key no
> +check_offlows_for_datapath hv2 $lr1_dp_key no
> +check_offlows_for_datapath hv2 $sw2_dp_key no
> +check_offlows_for_datapath hv2 $lr2_dp_key no
> +
> +check_offlows_for_datapath gw1 $sw0_dp_key yes
> +check_offlows_for_datapath gw1 $lr0_dp_key yes
> +check_offlows_for_datapath gw1 $public_dp_key yes
> +check_offlows_for_datapath gw1 $sw1_dp_key no
> +check_offlows_for_datapath gw1 $lr1_dp_key no
> +check_offlows_for_datapath gw1 $sw2_dp_key no
> +check_offlows_for_datapath gw1 $lr2_dp_key no
> +
> +check_offlows_for_datapath gw2 $sw0_dp_key no
> +check_offlows_for_datapath gw2 $lr0_dp_key no
> +check_offlows_for_datapath hv1 $public_dp_key yes
> +check_offlows_for_datapath gw2 $sw1_dp_key yes
> +check_offlows_for_datapath gw2 $lr1_dp_key yes
> +check_offlows_for_datapath gw2 $sw2_dp_key yes
> +check_offlows_for_datapath gw2 $lr2_dp_key yes
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw1, type: switch
> +Datapath: sw2, type: switch
> +Local datapaths:
> +])
> +
> +AS_BOX([Create a VIF on hv2 for sw1-port1])
> +
> +as hv2
> +ovs-vsctl -- add-port br-int hv2-vif1 -- \
> +    set interface hv2-vif1 external-ids:iface-id=sw1-port1 \
> +    options:tx_pcap=hv2/vif1-tx.pcap \
> +    options:rxq_pcap=hv2/vif1-rx.pcap \
> +    ofport-request=1
> +
> +wait_for_ports_up sw1-port1
> +
> +check_offlows_for_datapath hv1 $sw0_dp_key yes
> +check_offlows_for_datapath hv1 $lr0_dp_key yes
> +check_offlows_for_datapath hv1 $public_dp_key yes
> +check_offlows_for_datapath hv1 $sw1_dp_key no
> +check_offlows_for_datapath hv1 $lr1_dp_key no
> +check_offlows_for_datapath hv1 $sw2_dp_key no
> +check_offlows_for_datapath hv1 $lr2_dp_key no
> +
> +check_offlows_for_datapath hv2 $sw0_dp_key no
> +check_offlows_for_datapath hv2 $lr0_dp_key no
> +
> +# Since there are no distributed dnat_and_snat entries
> +# in lr1, hv2 will not have "public" in its
> +# local datapaths.
> +check_offlows_for_datapath hv2 $public_dp_key no
> +check_offlows_for_datapath hv2 $sw1_dp_key yes
> +check_offlows_for_datapath hv2 $lr1_dp_key yes
> +check_offlows_for_datapath hv2 $sw2_dp_key no
> +check_offlows_for_datapath hv2 $lr2_dp_key no
> +
> +check_offlows_for_datapath gw1 $sw0_dp_key yes
> +check_offlows_for_datapath gw1 $lr0_dp_key yes
> +check_offlows_for_datapath gw1 $public_dp_key yes
> +check_offlows_for_datapath gw1 $sw1_dp_key no
> +check_offlows_for_datapath gw1 $lr1_dp_key no
> +check_offlows_for_datapath gw1 $sw2_dp_key no
> +check_offlows_for_datapath gw1 $lr2_dp_key no
> +
> +# gw2 should have sw1, lr1, sw2 and lr2 and public in its local datapaths.
> +check_offlows_for_datapath gw2 $sw0_dp_key no
> +check_offlows_for_datapath gw2 $lr0_dp_key no
> +check_offlows_for_datapath gw2 $public_dp_key yes
> +check_offlows_for_datapath gw2 $sw1_dp_key yes
> +check_offlows_for_datapath gw2 $lr1_dp_key yes
> +check_offlows_for_datapath gw2 $sw2_dp_key yes
> +check_offlows_for_datapath gw2 $lr2_dp_key yes
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: sw1, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw1, type: switch
> +Datapath: sw2, type: switch
> +Local datapaths:
> +])
> +
> +# Add distributed dnat_and_snat in lr1.  hv2 should have
> +# public in its local datapaths.
> +AS_BOX([ Add distributed dnat_and_snat in lr1])
> +
> +check ovn-nbctl lr-nat-del lr1 dnat_and_snat
> +check ovn-nbctl --wait=hv lr-nat-add lr1 dnat_and_snat 172.168.0.140 20.0.0.3 sw1-port1 10:00:00:01:02:14
> +
> +check_offlows_for_datapath hv2 $sw0_dp_key no
> +check_offlows_for_datapath hv2 $lr0_dp_key no
> +check_offlows_for_datapath hv2 $public_dp_key yes
> +check_offlows_for_datapath hv2 $sw1_dp_key yes
> +check_offlows_for_datapath hv2 $lr1_dp_key yes
> +check_offlows_for_datapath hv2 $sw2_dp_key no
> +check_offlows_for_datapath hv2 $lr2_dp_key no
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: public, type: switch
> +Datapath: sw1, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw1, type: switch
> +Datapath: sw2, type: switch
> +Local datapaths:
> +])
> +
> +AS_BOX([Create a VIF on hv2 for sw0-port2])
> +
> +as hv2
> +ovs-vsctl -- add-port br-int hv2-vif2 -- \
> +    set interface hv2-vif2 external-ids:iface-id=sw0-port2 \
> +    options:tx_pcap=hv2/vif2-tx.pcap \
> +    options:rxq_pcap=hv2/vif2-rx.pcap \
> +    ofport-request=2
> +
> +wait_for_ports_up sw0-port2
> +
> +check_offlows_for_datapath hv2 $sw0_dp_key yes
> +check_offlows_for_datapath hv2 $lr0_dp_key yes
> +check_offlows_for_datapath hv2 $public_dp_key yes
> +check_offlows_for_datapath hv2 $sw1_dp_key yes
> +check_offlows_for_datapath hv2 $lr1_dp_key yes
> +check_offlows_for_datapath hv2 $sw2_dp_key no
> +check_offlows_for_datapath hv2 $lr2_dp_key no
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: lr1, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Datapath: sw1, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw1, type: switch
> +Datapath: sw2, type: switch
> +Local datapaths:
> +])
> +
> +AS_BOX([Delete the VIF for sw1-port1 in hv2])
> +
> +as hv2 ovs-vsctl del-port hv2-vif1
> +check ovn-nbctl --wait=hv sync
> +check_column "false" Port_Binding up logical_port=sw1-port1
> +
> +check_offlows_for_datapath hv2 $sw0_dp_key yes
> +check_offlows_for_datapath hv2 $lr0_dp_key yes
> +check_offlows_for_datapath hv2 $public_dp_key yes
> +check_offlows_for_datapath hv2 $sw1_dp_key no
> +check_offlows_for_datapath hv2 $lr1_dp_key no
> +check_offlows_for_datapath hv2 $sw2_dp_key no
> +check_offlows_for_datapath hv2 $lr2_dp_key no
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw1, type: switch
> +Datapath: sw2, type: switch
> +Local datapaths:
> +])
> +
> +AS_BOX([Delete the VIF for sw0-port2 in hv2])
> +
> +# Presently when a port binding is released we are not
> +# deleting its datapath from the local_datapaths if it
> +# is not relevant anymore.
> +
> +as hv2 ovs-vsctl del-port hv2-vif2
> +check ovn-nbctl --wait=hv sync
> +check_column "false" Port_Binding up logical_port=sw0-port2
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +# hv2 would still have public, sw0 and lr0 in its local datapaths.
> +# Next recompute should delete these datapaths.
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw1, type: switch
> +Datapath: sw2, type: switch
> +Local datapaths:
> +])
> +
> +# Trigger a recompute
> +AS_BOX([Trigger a recompute in hv2])
> +check as hv2 ovn-appctl inc-engine/recompute
> +
> +check_offlows_for_datapath hv2 $sw0_dp_key no
> +check_offlows_for_datapath hv2 $lr0_dp_key no
> +check_offlows_for_datapath hv2 $public_dp_key no
> +check_offlows_for_datapath hv2 $sw1_dp_key no
> +check_offlows_for_datapath hv2 $lr1_dp_key no
> +check_offlows_for_datapath hv2 $sw2_dp_key no
> +check_offlows_for_datapath hv2 $lr2_dp_key no
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw1, type: switch
> +Datapath: sw2, type: switch
> +Local datapaths:
> +])
> +
> +AS_BOX([Disconnect sw2 from lr2])
> +
> +check ovn-nbctl --wait=hv lsp-set-options sw2-lr2 router-port=lr2-sw2xxx
> +check_offlows_for_datapath hv1 $sw0_dp_key yes
> +check_offlows_for_datapath hv1 $lr0_dp_key yes
> +check_offlows_for_datapath hv1 $public_dp_key yes
> +check_offlows_for_datapath hv1 $sw1_dp_key no
> +check_offlows_for_datapath hv1 $lr1_dp_key no
> +check_offlows_for_datapath hv1 $sw2_dp_key no
> +check_offlows_for_datapath hv1 $lr2_dp_key no
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw1, type: switch
> +Local datapaths:
> +])
> +
> +AS_BOX([Reconnect sw2 to lr2 again])
> +
> +check ovn-nbctl --wait=hv lsp-set-options sw2-lr2 router-port=lr2-sw2
> +check_offlows_for_datapath hv1 $sw0_dp_key yes
> +check_offlows_for_datapath hv1 $lr0_dp_key yes
> +check_offlows_for_datapath hv1 $public_dp_key yes
> +check_offlows_for_datapath hv1 $sw1_dp_key no
> +check_offlows_for_datapath hv1 $lr1_dp_key no
> +check_offlows_for_datapath hv1 $sw2_dp_key no
> +check_offlows_for_datapath hv1 $lr2_dp_key no
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw1, type: switch
> +Datapath: sw2, type: switch
> +Local datapaths:
> +])
> +
> +AS_BOX([Create a VIF on gw2 for sw1-port1])
> +
> +as gw2
> +ovs-vsctl -- add-port br-int gw2-vif2 -- \
> +    set interface gw2-vif2 external-ids:iface-id=sw1-port1 \
> +    options:tx_pcap=gw2/vif2-tx.pcap \
> +    options:rxq_pcap=gw2/vif2-rx.pcap \
> +    ofport-request=2
> +
> +wait_for_ports_up sw1-port1
> +
> +check_offlows_for_datapath gw2 $sw0_dp_key no
> +check_offlows_for_datapath gw2 $lr0_dp_key no
> +check_offlows_for_datapath gw2 $public_dp_key yes
> +check_offlows_for_datapath gw2 $sw1_dp_key yes
> +check_offlows_for_datapath gw2 $lr1_dp_key yes
> +check_offlows_for_datapath gw2 $sw2_dp_key yes
> +check_offlows_for_datapath gw2 $lr2_dp_key yes
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw1, type: switch
> +Datapath: sw2, type: switch
> +Local datapaths:
> +])
> +
> +AS_BOX([Delete the VIF for sw1-port1 in gw2])
> +
> +as gw2 ovs-vsctl del-port gw2-vif2
> +check ovn-nbctl --wait=hv sync
> +check_column "false" Port_Binding up logical_port=sw1-port1
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +# hv2 would still have public in its local datapaths.  Next recompute should
> +# delete this datapath from the local datapaths.
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw1, type: switch
> +Datapath: sw2, type: switch
> +Local datapaths:
> +])
> +
> +AS_BOX([Create a logical port for public and bind it on hv2])
> +# hv2 will only have public in its local datapaths.
> +check ovn-nbctl lsp-add public public-p1
> +
> +as hv2
> +ovs-vsctl -- add-port br-int hv2-vif3 -- \
> +    set interface hv2-vif3 external-ids:iface-id=public-p1 \
> +    options:tx_pcap=hv2/vif3-tx.pcap \
> +    options:rxq_pcap=hv2/vif3-rx.pcap \
> +    ofport-request=2
> +
> +wait_for_ports_up public-p1
> +
> +# as hv2 ovn-appctl -t ovn-controller inc-engine/recompute
> +# check ovn-nbctl --wait=hv sync
> +
> +as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort
> +
> +check_offlows_for_datapath hv2 $sw0_dp_key no
> +check_offlows_for_datapath hv2 $lr0_dp_key no
> +check_offlows_for_datapath hv2 $public_dp_key yes
> +check_offlows_for_datapath hv2 $sw1_dp_key no
> +check_offlows_for_datapath hv2 $lr1_dp_key no
> +check_offlows_for_datapath hv2 $sw2_dp_key no
> +check_offlows_for_datapath hv2 $lr2_dp_key no
> +
> +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: public, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr0, type: router
> +Datapath: public, type: switch
> +Datapath: sw0, type: switch
> +Local datapaths:
> +])
> +
> +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> +Datapath: lr1, type: router
> +Datapath: lr2, type: router
> +Datapath: public, type: switch
> +Datapath: sw1, type: switch
> +Datapath: sw2, type: switch
> +Local datapaths:
> +])
> +
> +OVN_CLEANUP([hv1], [hv2], [gw1], [gw2])
> +AT_CLEANUP
> +])
> +
>  OVN_FOR_EACH_NORTHD([
>  AT_SETUP([requested-tnl-key-recompute])
>  AT_KEYWORDS([requested-tnl-key-recompute])
> -- 
> 2.48.1
> 
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev

On Tue, Feb 11, 2025 at 5:15 AM Felix Huettner
<felix.huettner@stackit.cloud> wrote:
>
> On Mon, Feb 10, 2025 at 10:37:25AM -0500, numans@ovn.org wrote:
> > From: Numan Siddique <numans@ovn.org>
> >
> > Consider the below logical topology
> >
> > sw0-p1 -
> >         |
> > sw0-p2 -   ->  sw0 -> lr0 ----
> > ...     |                     |
> > sw0-pn -                      |
> >                               |
> > sw1-p1 -                      |
> >         |                     |
> > sw1p-2 -   ->  sw1 -> lr1 ----  --- public (provider switch)
> > ...     |                     |
> > sw1-pn-                       |
> >                               |
> > swn-p1 -                      |
> >         |                     |
> > swn-p2-    ->  swn -> lrn ----
> > ...     |
> > swn-pn -
> >
> > All the routers are connected to the provider switch via
> > a ditributed gateway port.
> >
> > If sw0-p1 is resident on the chassis C1, then since there is a path
> > to all the switches and the routers, ovn-controller will add all
> > these datapaths to its 'local_datapaths' map.  This in turn results
> > in processing all the logical flows and installing all the openflows
> > and in turn wasting the CPU time.  This can be very costly in
> > a highly scaled deployment.
> >
> > Previous commit sets a flag "only_dgp_peer_ports" in the SB Datapath
> > binding for a provider switch (with only dgp peer ports).
> >
> > In this commit, ovn-controller makes use of this flag and stops
> > adding other datapaths connected to the public provider switch
> > to the 'local_datapaths'.
> >
> > For example, when it claims sw0-p1, it adds sw0, lr0 and public
> > to the local_datapaths and stops there.  If it later claims
> > sw1-p1, it will add sw1 and lr1.
> >
> > This reduces the recompute time and the number of openflow rules
> > added to ovs-vswitchd significantly.
> >
> > I tested this patch with a deployment of below logical resources:
> >
> > No of logical switches - 778
> > No of logical routers  - 871
> > No of logical flows    - 85626
> > No of 'ovn-sbctl dump-flows' - 208631
> >
> > Without this patch, afte claiming sw0-p1, ovn-controller adds
> > 269098 openflow rules and it takes approx 2500 milli seconds
> > for a recompute.
> >
> > With this patch, after claiming sw0-p1, ovn-controller adds
> > 21350 openflow rules and it takes approx 280 milli seconds
> > for a recompute.
> >
> > There is approx 90% reduction in the openflow rules and
> > 88% reduction in recompute time when a comoute node has
> > VIFs from one logical switch.
>
> Hi Numan,
>
> that sounds really great. We have a similar setup that would heavily
> benefit from this.
>
> However when reading through the patch i have one concern about the
> correctness of not considering these ports, but that might just come
> from misunderstanding something.
>
> I assumed that we have one chassis c1 that has claimed sw0-p1. It should
> then know sw0, lr0 and public.
> I also assumed that there is chassis c2 claiming sw1-p1. That would then
> know sw1, lr1 and public.
> Then i assumed there is a chassis g1 that has claimed the lr0 CR port.
> Also chassis g2 has claimed the lr1 CR port.
>
> Now we add a NAT rule to lr0. It is configured with dnat_and_snat and
> the settings necessary to process it in a distributed way (i think by
> setting logical_port and stateless, but i may be wrong here).

It's actually logical_port and external_mac

It maps
> the sw0-p1 IP to some IP on public.
>
> On lr1 we have a similar setup of NAT entries for the sw1-p1 IP.
>
> If i understand these NAT settins correctly this should now lead to traffic
> from sw0-p1 to sw1-p1 to flow directly (when using the external ip of the
> sw1-p1 NAT rule). So the traffic would flow from chassic c1 to c2
> without any involvement of g1 and g2.
>
That's correct.

> In this case wouldn't chassis c1 need to know about the lr1 and sw1
> datapath?

Not really.  Packet from sw0-p1 on C1 will enter lr0 pipeline first.
If there is a mac_binding entry
for the public IP of sw1-p1 in the lr0 datapath, it will set the
eth.dst to this mac and sends
the packet to public switch datapath and then to the wire via the localnet port.
The fabric will make sure that the packet reaches C2.
If there is no mac_binding entry, ovn-controller will generate an ARP
request for the sw1-p1 public IP.
And then eventually reinjects the packet after it learns the mac.

The same happens on the C2 for the reply path.

Even without this patch,  I think this is the behavior.

There is a multinode system test to cover this scenario.

Hope this addresses your concern.  Let me know if you have further questions.

Numan

>
> I guess i missed something somewhere, but since i am not sure i wanted
> to share this concern.
>
> Thanks a lot,
> Felix
>
> >
> > Signed-off-by: Numan Siddique <numans@ovn.org>
> > ---
> >  controller/binding.c        | 249 +++++++++--
> >  controller/binding.h        |   2 +
> >  controller/local_data.c     |  84 +++-
> >  controller/local_data.h     |   6 +
> >  controller/lport.c          |  12 +
> >  controller/lport.h          |   4 +
> >  controller/ovn-controller.c |  38 ++
> >  tests/multinode.at          | 178 ++++++++
> >  tests/ovn-performance.at    |   6 +-
> >  tests/ovn.at                | 853 ++++++++++++++++++++++++++++++++++++
> >  10 files changed, 1394 insertions(+), 38 deletions(-)
> >
> > diff --git a/controller/binding.c b/controller/binding.c
> > index ea5bf5a9fd..88c9640c3a 100644
> > --- a/controller/binding.c
> > +++ b/controller/binding.c
> > @@ -825,6 +825,16 @@ static bool binding_lport_update_port_sec(
> >  static bool ovs_iface_matches_lport_iface_id_ver(
> >      const struct ovsrec_interface *,
> >      const struct sbrec_port_binding *);
> > +static bool cleanup_patch_port_local_dps(
> > +    const struct sbrec_port_binding *, const struct sbrec_port_binding *cr_pb,
> > +    const struct sbrec_port_binding *peer, struct local_datapath *ld,
> > +    struct binding_ctx_in *b_ctx_in,
> > +    struct binding_ctx_out *b_ctx_out,
> > +    bool *cleanup);
> > +static bool local_datapath_is_relevant(
> > +    struct local_datapath *, struct local_datapath *ignore_peer_ld,
> > +    struct hmap *local_datapaths, int *depth, const struct sbrec_chassis *,
> > +    struct ovsdb_idl_index *);
> >
> >  void
> >  related_lports_init(struct related_lports *rp)
> > @@ -1062,6 +1072,13 @@ binding_dump_related_lports(struct related_lports *related_lports,
> >      }
> >  }
> >
> > +struct dp_binding {
> > +    struct hmap_node key_node;
> > +
> > +    uint32_t dp_key;
> > +    struct hmapx binding_lports;
> > +};
> > +
> >  void
> >  binding_dump_local_bindings(struct local_binding_data *lbinding_data,
> >                              struct ds *out_data)
> > @@ -1130,6 +1147,19 @@ binding_dump_local_bindings(struct local_binding_data *lbinding_data,
> >      free(nodes);
> >  }
> >
> > +void
> > +binding_dump_local_datapaths(struct hmap *local_datapaths,
> > +                             struct ds *out_data)
> > +{
> > +    ds_put_cstr(out_data, "Local datapaths:\n");
> > +    struct local_datapath *ld;
> > +    HMAP_FOR_EACH (ld, hmap_node, local_datapaths) {
> > +        ds_put_format(out_data, "Datapath: %s, type: %s\n",
> > +                      smap_get(&ld->datapath->external_ids, "name"),
> > +                      ld->is_switch ? "switch" : "router");
> > +    }
> > +}
> > +
> >  void
> >  set_pb_chassis_in_sbrec(const struct sbrec_port_binding *pb,
> >                          const struct sbrec_chassis *chassis_rec,
> > @@ -2137,7 +2167,9 @@ build_local_bindings(struct binding_ctx_in *b_ctx_in,
> >
> >  static bool consider_patch_port_for_local_datapaths(
> >          const struct sbrec_port_binding *,
> > -        struct binding_ctx_in *, struct binding_ctx_out *);
> > +        const struct sbrec_port_binding *cr_pb,
> > +        struct binding_ctx_in *, struct binding_ctx_out *,
> > +        bool check_and_remove_localdps);
> >
> >  void
> >  binding_run(struct binding_ctx_in *b_ctx_in, struct binding_ctx_out *b_ctx_out)
> > @@ -2182,7 +2214,8 @@ binding_run(struct binding_ctx_in *b_ctx_in, struct binding_ctx_out *b_ctx_out)
> >          switch (lport_type) {
> >          case LP_PATCH:
> >              update_related_lport(pb, b_ctx_out);
> > -            consider_patch_port_for_local_datapaths(pb, b_ctx_in, b_ctx_out);
> > +            consider_patch_port_for_local_datapaths(pb, NULL, b_ctx_in,
> > +                                                    b_ctx_out, false);
> >              break;
> >
> >          case LP_VTEP:
> > @@ -2238,6 +2271,9 @@ binding_run(struct binding_ctx_in *b_ctx_in, struct binding_ctx_out *b_ctx_out)
> >              struct lport *lnet_lport = xmalloc(sizeof *lnet_lport);
> >              lnet_lport->pb = pb;
> >              ovs_list_push_back(&localnet_lports, &lnet_lport->list_node);
> > +            if (pb->chassis == b_ctx_in->chassis_rec) {
> > +                sbrec_port_binding_set_chassis(pb, NULL);
> > +            }
> >              break;
> >          }
> >
> > @@ -2566,7 +2602,6 @@ consider_iface_release(const struct ovsrec_interface *iface_rec,
> >                                             lbinding->iface->name,
> >                                             &lbinding->iface->header_.uuid);
> >              }
> > -
> >          } else if (b_lport && b_lport->type == LP_LOCALPORT) {
> >              /* lbinding is associated with a localport.  Remove it from the
> >               * related lports. */
> > @@ -2956,12 +2991,27 @@ handle_updated_vif_lport(const struct sbrec_port_binding *pb,
> >
> >  static bool
> >  consider_patch_port_for_local_datapaths(const struct sbrec_port_binding *pb,
> > +                                        const struct sbrec_port_binding *cr_pb,
> >                                          struct binding_ctx_in *b_ctx_in,
> > -                                        struct binding_ctx_out *b_ctx_out)
> > +                                        struct binding_ctx_out *b_ctx_out,
> > +                                        bool check_and_remove_localdps)
> >  {
> > -    struct local_datapath *ld =
> > -        get_local_datapath(b_ctx_out->local_datapaths,
> > -                           pb->datapath->tunnel_key);
> > +    const struct sbrec_port_binding *peer;
> > +    struct local_datapath *peer_ld = NULL;
> > +    struct local_datapath *ld = NULL;
> > +
> > +    ld = get_local_datapath(b_ctx_out->local_datapaths,
> > +                            pb->datapath->tunnel_key);
> > +    if (ld && ld->has_only_dgp_peer_ports) {
> > +        /* Nothing much to do. */
> > +        return true;
> > +    }
> > +
> > +    peer = lport_get_peer(pb, b_ctx_in->sbrec_port_binding_by_name);
> > +    if (peer) {
> > +        peer_ld = get_local_datapath(b_ctx_out->local_datapaths,
> > +                                     peer->datapath->tunnel_key);
> > +    }
> >
> >      if (!ld) {
> >          /* If 'ld' for this lport is not present, then check if
> > @@ -2969,17 +3019,9 @@ consider_patch_port_for_local_datapaths(const struct sbrec_port_binding *pb,
> >           * and peer's datapath is already in the local datapaths,
> >           * then add this lport's datapath to the local_datapaths.
> >           * */
> > -        const struct sbrec_port_binding *peer;
> > -        struct local_datapath *peer_ld = NULL;
> > -        peer = lport_get_peer(pb, b_ctx_in->sbrec_port_binding_by_name);
> > -        if (peer) {
> > -            peer_ld =
> > -                get_local_datapath(b_ctx_out->local_datapaths,
> > -                                   peer->datapath->tunnel_key);
> > -        }
> > -        if (peer_ld && need_add_peer_to_local(
> > -                b_ctx_in->sbrec_port_binding_by_name, peer,
> > -                b_ctx_in->chassis_rec)) {
> > +        if (peer_ld && !peer_ld->has_only_dgp_peer_ports &&
> > +            need_add_peer_to_local(b_ctx_in->sbrec_port_binding_by_name, peer,
> > +                                   b_ctx_in->chassis_rec)) {
> >              ld = add_local_datapath(
> >                      b_ctx_in->sbrec_datapath_binding_by_key,
> >                      b_ctx_in->sbrec_port_binding_by_datapath,
> > @@ -2992,7 +3034,7 @@ consider_patch_port_for_local_datapaths(const struct sbrec_port_binding *pb,
> >          /* Add the peer datapath to the local datapaths if it's
> >           * not present yet.
> >           */
> > -        if (need_add_peer_to_local(
> > +        if (peer && need_add_peer_to_local(
> >                  b_ctx_in->sbrec_port_binding_by_name, pb,
> >                  b_ctx_in->chassis_rec)) {
> >              add_local_datapath_peer_port(
> > @@ -3003,6 +3045,18 @@ consider_patch_port_for_local_datapaths(const struct sbrec_port_binding *pb,
> >                  ld, b_ctx_out->local_datapaths,
> >                  b_ctx_out->tracked_dp_bindings);
> >          }
> > +
> > +        if (check_and_remove_localdps) {
> > +            bool cleanedup = false;
> > +            if (!cleanup_patch_port_local_dps(pb, cr_pb, peer, ld, b_ctx_in,
> > +                                              b_ctx_out, &cleanedup)) {
> > +                return false;
> > +            }
> > +
> > +            if (cleanedup) {
> > +                ld = NULL;
> > +            }
> > +        }
> >      }
> >
> >      /* If this chassis is requested - try to claim. */
> > @@ -3021,12 +3075,10 @@ consider_patch_port_for_local_datapaths(const struct sbrec_port_binding *pb,
> >          || if_status_is_port_claimed(b_ctx_out->if_mgr, pb->logical_port)) {
> >
> >          remove_local_lports(pb->logical_port, b_ctx_out);
> > -        if (!release_lport(pb, ld, b_ctx_in->chassis_rec,
> > -                           !b_ctx_in->ovnsb_idl_txn,
> > -                           b_ctx_out->tracked_dp_bindings,
> > -                           b_ctx_out->if_mgr)) {
> > -            return false;
> > -        }
> > +        return release_lport(pb, ld, b_ctx_in->chassis_rec,
> > +                             !b_ctx_in->ovnsb_idl_txn,
> > +                             b_ctx_out->tracked_dp_bindings,
> > +                             b_ctx_out->if_mgr);
> >      }
> >      return true;
> >  }
> > @@ -3086,8 +3138,8 @@ handle_updated_port(struct binding_ctx_in *b_ctx_in,
> >
> >      case LP_PATCH:
> >          update_related_lport(pb, b_ctx_out);
> > -        handled = consider_patch_port_for_local_datapaths(pb, b_ctx_in,
> > -                                                              b_ctx_out);
> > +        handled = consider_patch_port_for_local_datapaths(pb, NULL, b_ctx_in,
> > +                                                          b_ctx_out, true);
> >          break;
> >
> >      case LP_VTEP:
> > @@ -3130,8 +3182,8 @@ handle_updated_port(struct binding_ctx_in *b_ctx_in,
> >              break;
> >          }
> >          handled = consider_patch_port_for_local_datapaths(distributed_pb,
> > -                                                          b_ctx_in,
> > -                                                          b_ctx_out);
> > +                                                          pb, b_ctx_in,
> > +                                                          b_ctx_out, true);
> >          break;
> >
> >      case LP_EXTERNAL:
> > @@ -3901,3 +3953,142 @@ binding_destroy(void)
> >      shash_destroy_free_data(&_qos_ports);
> >      sset_clear(&_postponed_ports);
> >  }
> > +
> > +static bool
> > +is_patch_pb_chassis_relevant(
> > +    const struct sbrec_port_binding *pb,
> > +    const struct sbrec_chassis *chassis,
> > +    struct ovsdb_idl_index *sbrec_port_binding_by_name)
> > +{
> > +    if (ha_chassis_group_contains(pb->ha_chassis_group, chassis)) {
> > +        return true;
> > +    }
> > +
> > +    const struct sbrec_port_binding *pb_crp =
> > +        lport_get_cr_port(sbrec_port_binding_by_name, pb);
> > +    if (pb_crp) {
> > +        return ha_chassis_group_contains(pb_crp->ha_chassis_group, chassis);
> > +    }
> > +
> > +    return false;
> > +}
> > +
> > +static bool
> > +cleanup_patch_port_local_dps(const struct sbrec_port_binding *pb,
> > +                             const struct sbrec_port_binding *cr_pb,
> > +                             const struct sbrec_port_binding *peer,
> > +                             struct local_datapath *ld,
> > +                             struct binding_ctx_in *b_ctx_in,
> > +                             struct binding_ctx_out *b_ctx_out,
> > +                             bool *cleanedup)
> > +{
> > +    *cleanedup = false;
> > +    if (!peer) {
> > +        /* Remove 'pb' from the ld's peer ports as it has no peer. */
> > +        remove_local_datapath_peer_port(pb, ld,
> > +                                        b_ctx_out->local_datapaths);
> > +    }
> > +
> > +    /* We can consider removing the 'ld' of the patch port 'pb' from the
> > +     * local datapaths, if all the below conditions are met
> > +     *     - 'pb' doesn't have a peer or ld' is a router datapath
> > +     *     - if 'pb' is a distributed gateway port (dgp), then
> > +     *       its chassisredirect port's ha chassis group doesn't
> > +     *       contain our 'chassis rec'
> > +     *     - and finally 'ld' is not relevant any more.  See
> > +     *       local_datapath_is_relevant() for more details.
> > +     *
> > +     * Note: If 'ld' can be removed, then all its connected local datapaths
> > +     * can also be removed.
> > +     *
> > +     * For example, if we had sw1-port1 ->  sw1 -> lr1 -> sw2 and if
> > +     * sw1-port1 resides on this chassis, and if the link between sw1 and
> > +     * lr1 is broken, then we can remove lr1 and sw2 from the
> > +     * local_datapaths.
> > +     * */
> > +
> > +    bool consider_ld_for_removal = !peer || !ld->is_switch;
> > +    if (consider_ld_for_removal && cr_pb) {
> > +        consider_ld_for_removal = !ha_chassis_group_contains(
> > +            cr_pb->ha_chassis_group, b_ctx_in->chassis_rec);
> > +    }
> > +
> > +    if (!consider_ld_for_removal) {
> > +        return true;
> > +    }
> > +
> > +    int depth = 0;
> > +
> > +    bool is_relevant = local_datapath_is_relevant(
> > +        ld, NULL, b_ctx_out->local_datapaths,
> > +        &depth, b_ctx_in->chassis_rec,
> > +        b_ctx_in->sbrec_port_binding_by_name);
> > +
> > +    if (depth >= 100) {
> > +        /* datapaths are too deeply nested.  Fall back to recompute. */
> > +        return false;
> > +    }
> > +
> > +    if (!is_relevant) {
> > +        /* This 'ld' can be removed from the local datapaths as
> > +            *   - its a router datapath and
> > +            *   - it has no peers locally. */
> > +        local_datapath_remove_and_destroy(ld, b_ctx_out->local_datapaths,
> > +                                          b_ctx_out->tracked_dp_bindings);
> > +        *cleanedup = true;
> > +    }
> > +
> > +    return true;
> > +}
> > +
> > +static bool
> > +local_datapath_is_relevant(struct local_datapath *ld,
> > +                           struct local_datapath *ignore_peer_ld,
> > +                           struct hmap *local_datapaths, int *depth,
> > +                           const struct sbrec_chassis *chassis,
> > +                           struct ovsdb_idl_index *sbrec_pb_by_name)
> > +{
> > +    if (!sset_is_empty(&ld->claimed_lports) ||
> > +        !shash_is_empty(&ld->external_ports) ||
> > +        !shash_is_empty(&ld->multichassis_ports) ||
> > +        ld->vtep_port) {
> > +        return true;
> > +    }
> > +
> > +    bool relevant = false;
> > +
> > +    if (*depth >= 100) {
> > +        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> > +        VLOG_WARN_RL(&rl, "datapaths nested too deep");
> > +        return true;
> > +    }
> > +
> > +    for (size_t i = 0; i < ld->n_peer_ports && !relevant; i++) {
> > +        const struct sbrec_port_binding *remote = ld->peer_ports[i].remote;
> > +        const struct sbrec_port_binding *local = ld->peer_ports[i].local;
> > +
> > +        if (is_patch_pb_chassis_relevant(local, chassis,
> > +                                         sbrec_pb_by_name)) {
> > +            return  true;
> > +        }
> > +
> > +        if (is_patch_pb_chassis_relevant(remote, chassis,
> > +                                         sbrec_pb_by_name)) {
> > +            return  true;
> > +        }
> > +
> > +        struct local_datapath *peer_ld;
> > +        uint32_t remote_peer_ld_key;
> > +        remote_peer_ld_key = ld->peer_ports[i].remote->datapath->tunnel_key;
> > +        peer_ld = get_local_datapath(local_datapaths, remote_peer_ld_key);
> > +        if (peer_ld && !peer_ld->has_only_dgp_peer_ports &&
> > +            peer_ld != ignore_peer_ld) {
> > +            *depth = *depth + 1;
> > +            relevant = local_datapath_is_relevant(peer_ld, ld,
> > +                                                  local_datapaths, depth,
> > +                                                  chassis, sbrec_pb_by_name);
> > +        }
> > +    }
> > +
> > +    return relevant;
> > +}
> > diff --git a/controller/binding.h b/controller/binding.h
> > index d13ae36c79..a4346c3e10 100644
> > --- a/controller/binding.h
> > +++ b/controller/binding.h
> > @@ -201,6 +201,8 @@ bool binding_handle_port_binding_changes(struct binding_ctx_in *,
> >  void binding_tracked_dp_destroy(struct hmap *tracked_datapaths);
> >
> >  void binding_dump_local_bindings(struct local_binding_data *, struct ds *);
> > +void binding_dump_local_datapaths(struct hmap *local_datapaths,
> > +                                  struct ds *out_data);
> >
> >  void binding_dump_related_lports(struct related_lports *related_lports,
> >                                   struct ds *);
> > diff --git a/controller/local_data.c b/controller/local_data.c
> > index e19b2bf865..bb5740f1d6 100644
> > --- a/controller/local_data.c
> > +++ b/controller/local_data.c
> > @@ -53,6 +53,13 @@ static struct tracked_datapath *tracked_datapath_create(
> >
> >  static bool datapath_is_switch(const struct sbrec_datapath_binding *);
> >  static bool datapath_is_transit_switch(const struct sbrec_datapath_binding *);
> > +static bool datapath_has_only_dgp_peer_ports(
> > +    const struct sbrec_datapath_binding *);
> > +static void local_datapath_remove_and_destroy__(
> > +    struct local_datapath *ld,
> > +    const struct sbrec_port_binding *ignore_peer_port,
> > +    struct hmap *local_datapaths,
> > +    struct hmap *tracked_datapaths);
> >
> >  static uint64_t local_datapath_usage;
> >
> > @@ -86,6 +93,7 @@ local_datapath_alloc(const struct sbrec_datapath_binding *dp)
> >      ld->datapath = dp;
> >      ld->is_switch = datapath_is_switch(dp);
> >      ld->is_transit_switch = datapath_is_transit_switch(dp);
> > +    ld->has_only_dgp_peer_ports = datapath_has_only_dgp_peer_ports(dp);
> >      shash_init(&ld->external_ports);
> >      shash_init(&ld->multichassis_ports);
> >      sset_init(&ld->claimed_lports);
> > @@ -132,6 +140,14 @@ local_datapath_destroy(struct local_datapath *ld)
> >      free(ld);
> >  }
> >
> > +void local_datapath_remove_and_destroy(struct local_datapath *ld,
> > +                                       struct hmap *local_datapaths,
> > +                                       struct hmap *tracked_datapaths)
> > +{
> > +    local_datapath_remove_and_destroy__(ld, NULL, local_datapaths,
> > +                                        tracked_datapaths);
> > +}
> > +
> >  /* Checks if pb is running on local gw router or pb is a patch port
> >   * and the peer datapath should be added to local datapaths. */
> >  bool
> > @@ -226,12 +242,12 @@ add_local_datapath_peer_port(
> >          get_local_datapath(local_datapaths,
> >                             peer->datapath->tunnel_key);
> >      if (!peer_ld) {
> > -        add_local_datapath__(sbrec_datapath_binding_by_key,
> > -                             sbrec_port_binding_by_datapath,
> > -                             sbrec_port_binding_by_name, 1,
> > -                             peer->datapath, chassis, local_datapaths,
> > -                             tracked_datapaths);
> > -        return;
> > +        peer_ld = add_local_datapath__(sbrec_datapath_binding_by_key,
> > +                                       sbrec_port_binding_by_datapath,
> > +                                       sbrec_port_binding_by_name, 1,
> > +                                       peer->datapath, chassis,
> > +                                       local_datapaths,
> > +                                       tracked_datapaths);
> >      }
> >
> >      local_datapath_peer_port_add(peer_ld, peer, pb);
> > @@ -613,6 +629,17 @@ add_local_datapath__(struct ovsdb_idl_index *sbrec_datapath_binding_by_key,
> >                               tracked_datapaths);
> >      }
> >
> > +    if (ld->has_only_dgp_peer_ports) {
> > +        /* If this flag is set, it means this 'switch' datapath has
> > +         *  - one ore many localnet ports.
> > +         *  - all the router ports it is connected to are
> > +         *    distributed gateway ports (DGPs).
> > +         * There is no need to add the routers of the dgps to
> > +         * the local datapaths.
> > +         * */
> > +        return ld;
> > +    }
> > +
> >      if (depth >= 100) {
> >          static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> >          VLOG_WARN_RL(&rl, "datapaths nested too deep");
> > @@ -705,6 +732,13 @@ datapath_is_transit_switch(const struct sbrec_datapath_binding *ldp)
> >      return smap_get(&ldp->external_ids, "interconn-ts") != NULL;
> >  }
> >
> > +static bool
> > +datapath_has_only_dgp_peer_ports(const struct sbrec_datapath_binding *ldp)
> > +{
> > +    return datapath_is_switch(ldp) &&
> > +           smap_get_bool(&ldp->external_ids, "only_dgp_peer_ports", false);
> > +}
> > +
> >  bool
> >  lb_is_local(const struct sbrec_load_balancer *sbrec_lb,
> >              const struct hmap *local_datapaths)
> > @@ -745,3 +779,41 @@ lb_is_local(const struct sbrec_load_balancer *sbrec_lb,
> >
> >      return false;
> >  }
> > +
> > +static void
> > +local_datapath_remove_and_destroy__(struct local_datapath *ld,
> > +                                    const struct sbrec_port_binding *ignore_pb,
> > +                                    struct hmap *local_datapaths,
> > +                                    struct hmap *tracked_datapaths)
> > +{
> > +    for (size_t i = 0; i < ld->n_peer_ports; i++) {
> > +        const struct sbrec_port_binding *remote = ld->peer_ports[i].remote;
> > +        const struct sbrec_port_binding *local = ld->peer_ports[i].local;
> > +
> > +        if (local == ignore_pb) {
> > +            continue;
> > +        }
> > +
> > +        struct local_datapath *peer_ld;
> > +        uint32_t remote_peer_ld_key;
> > +
> > +        remote_peer_ld_key = ld->peer_ports[i].remote->datapath->tunnel_key;
> > +        peer_ld = get_local_datapath(local_datapaths, remote_peer_ld_key);
> > +        if (peer_ld && !peer_ld->has_only_dgp_peer_ports) {
> > +            local_datapath_remove_and_destroy__(peer_ld, remote,
> > +                                                local_datapaths,
> > +                                                tracked_datapaths);
> > +        } else if (peer_ld && peer_ld->has_only_dgp_peer_ports) {
> > +            remove_local_datapath_peer_port(ld->peer_ports[i].remote,
> > +                                            peer_ld, local_datapaths);
> > +        }
> > +    }
> > +
> > +    hmap_remove(local_datapaths, &ld->hmap_node);
> > +    if (tracked_datapaths) {
> > +        tracked_datapath_add(ld->datapath, TRACKED_RESOURCE_REMOVED,
> > +                             tracked_datapaths);
> > +    }
> > +
> > +    local_datapath_destroy(ld);
> > +}
> > diff --git a/controller/local_data.h b/controller/local_data.h
> > index 857d63a51d..ef01f4259e 100644
> > --- a/controller/local_data.h
> > +++ b/controller/local_data.h
> > @@ -46,6 +46,8 @@ struct local_datapath {
> >      const struct sbrec_datapath_binding *datapath;
> >      bool is_switch;
> >      bool is_transit_switch;
> > +    /* Valid only for 'is_switch' local datapath. */
> > +    bool has_only_dgp_peer_ports;
> >
> >      /* The localnet port in this datapath, if any (at most one is allowed). */
> >      const struct sbrec_port_binding *localnet_port;
> > @@ -91,6 +93,10 @@ struct local_datapath * add_local_datapath(
> >
> >  void local_datapaths_destroy(struct hmap *local_datapaths);
> >  void local_datapath_destroy(struct local_datapath *ld);
> > +void local_datapath_remove_and_destroy(struct local_datapath *,
> > +                                       struct hmap *local_datapaths,
> > +                                       struct hmap *tracked_datapaths);
> > +
> >  void add_local_datapath_peer_port(
> >      const struct sbrec_port_binding *,
> >      const struct sbrec_chassis *,
> > diff --git a/controller/lport.c b/controller/lport.c
> > index 8bc230e896..f164803528 100644
> > --- a/controller/lport.c
> > +++ b/controller/lport.c
> > @@ -102,6 +102,18 @@ lport_get_l3gw_peer(const struct sbrec_port_binding *pb,
> >      return get_peer_lport(pb, sbrec_port_binding_by_name);
> >  }
> >
> > +const struct sbrec_port_binding *
> > +lport_get_cr_port(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> > +                  const struct sbrec_port_binding *pb)
> > +{
> > +    const char *crp = smap_get(&pb->options, "chassis-redirect-port");
> > +    if (crp) {
> > +        return lport_lookup_by_name(sbrec_port_binding_by_name, crp);
> > +    }
> > +
> > +    return NULL;
> > +}
> > +
> >  enum can_bind
> >  lport_can_bind_on_this_chassis(const struct sbrec_chassis *chassis_rec,
> >                                 const struct sbrec_port_binding *pb)
> > diff --git a/controller/lport.h b/controller/lport.h
> > index 3a58ebe39e..069bd59096 100644
> > --- a/controller/lport.h
> > +++ b/controller/lport.h
> > @@ -73,4 +73,8 @@ const struct sbrec_port_binding *lport_get_l3gw_peer(
> >  bool
> >  lport_is_activated_by_activation_strategy(const struct sbrec_port_binding *pb,
> >                                            const struct sbrec_chassis *chassis);
> > +const struct sbrec_port_binding *lport_get_cr_port(
> > +    struct ovsdb_idl_index *sbrec_port_binding_by_name,
> > +    const struct sbrec_port_binding *);
> > +
> >  #endif /* controller/lport.h */
> > diff --git a/controller/ovn-controller.c b/controller/ovn-controller.c
> > index da942abaa9..dc8ceec076 100644
> > --- a/controller/ovn-controller.c
> > +++ b/controller/ovn-controller.c
> > @@ -101,6 +101,7 @@ static unixctl_cb_func debug_pause_execution;
> >  static unixctl_cb_func debug_resume_execution;
> >  static unixctl_cb_func debug_status_execution;
> >  static unixctl_cb_func debug_dump_local_bindings;
> > +static unixctl_cb_func debug_dump_local_datapaths;
> >  static unixctl_cb_func debug_dump_related_lports;
> >  static unixctl_cb_func debug_dump_local_template_vars;
> >  static unixctl_cb_func debug_dump_local_mac_bindings;
> > @@ -1688,6 +1689,22 @@ runtime_data_sb_datapath_binding_handler(struct engine_node *node OVS_UNUSED,
> >                  return false;
> >              }
> >          }
> > +
> > +        if (sbrec_datapath_binding_is_updated(
> > +                dp, SBREC_DATAPATH_BINDING_COL_EXTERNAL_IDS) &&
> > +            !sbrec_datapath_binding_is_new(dp)) {
> > +            struct local_datapath *ld =
> > +                get_local_datapath(&rt_data->local_datapaths,
> > +                                   dp->tunnel_key);
> > +                if (ld && ld->is_switch) {
> > +                    bool only_dgp_peer_ports =
> > +                        smap_get_bool(&dp->external_ids, "only_dgp_peer_ports",
> > +                                      false);
> > +                    if (ld->has_only_dgp_peer_ports != only_dgp_peer_ports) {
> > +                        return false;
> > +                    }
> > +                }
> > +        }
> >      }
> >
> >      return true;
> > @@ -4357,6 +4374,12 @@ lflow_output_runtime_data_handler(struct engine_node *node,
> >      init_lflow_ctx(node, fo, &l_ctx_in, &l_ctx_out);
> >
> >      struct tracked_datapath *tdp;
> > +    HMAP_FOR_EACH (tdp, node, tracked_dp_bindings) {
> > +        if (tdp->tracked_type == TRACKED_RESOURCE_REMOVED) {
> > +            return false;
> > +        }
> > +    }
> > +
> >      HMAP_FOR_EACH (tdp, node, tracked_dp_bindings) {
> >          if (tdp->tracked_type == TRACKED_RESOURCE_NEW) {
> >              if (!lflow_add_flows_for_datapath(tdp->dp, &l_ctx_in,
> > @@ -5552,6 +5575,10 @@ main(int argc, char *argv[])
> >                               debug_dump_local_bindings,
> >                               &runtime_data->lbinding_data);
> >
> > +    unixctl_command_register("debug/dump-local-datapaths", "", 0, 0,
> > +                             debug_dump_local_datapaths,
> > +                             &runtime_data->local_datapaths);
> > +
> >      unixctl_command_register("debug/dump-related-ports", "", 0, 0,
> >                               debug_dump_related_lports,
> >                               &runtime_data->related_lports);
> > @@ -6517,6 +6544,17 @@ debug_dump_local_bindings(struct unixctl_conn *conn, int argc OVS_UNUSED,
> >      ds_destroy(&binding_data);
> >  }
> >
> > +static void
> > +debug_dump_local_datapaths(struct unixctl_conn *conn, int argc OVS_UNUSED,
> > +                           const char *argv[] OVS_UNUSED,
> > +                           void *local_datapaths)
> > +{
> > +    struct ds local_dps_data = DS_EMPTY_INITIALIZER;
> > +    binding_dump_local_datapaths(local_datapaths, &local_dps_data);
> > +    unixctl_command_reply(conn, ds_cstr(&local_dps_data));
> > +    ds_destroy(&local_dps_data);
> > +}
> > +
> >  static void
> >  debug_dump_related_lports(struct unixctl_conn *conn, int argc OVS_UNUSED,
> >                            const char *argv[] OVS_UNUSED, void *related_lports)
> > diff --git a/tests/multinode.at b/tests/multinode.at
> > index c1bd3123ac..b828f16135 100644
> > --- a/tests/multinode.at
> > +++ b/tests/multinode.at
> > @@ -2776,3 +2776,181 @@ for i in 1 2; do
> >  done
> >
> >  AT_CLEANUP
> > +
> > +AT_SETUP([ovn multinode - only_dgp_peer_ports provider switch functionality])
> > +
> > +# Check that ovn-fake-multinode setup is up and running
> > +check_fake_multinode_setup
> > +
> > +# Delete the multinode NB and OVS resources before starting the test.
> > +cleanup_multinode_resources
> > +
> > +check multinode_nbctl ls-add sw0
> > +check multinode_nbctl lsp-add sw0 sw0-port1
> > +check multinode_nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03 10.0.0.3 1000::3"
> > +check multinode_nbctl lsp-add sw0 sw0-port2
> > +check multinode_nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:04 10.0.0.4 1000::4"
> > +
> > +check multinode_nbctl lr-add lr0
> > +check multinode_nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 1000::1/64
> > +check multinode_nbctl lsp-add sw0 sw0-lr0
> > +check multinode_nbctl lsp-set-type sw0-lr0 router
> > +check multinode_nbctl lsp-set-addresses sw0-lr0 router
> > +check multinode_nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> > +
> > +check multinode_nbctl ls-add public
> > +check multinode_nbctl lrp-add lr0 lr0-public 00:00:20:20:12:13 172.16.1.100/24 2000::1/64
> > +check multinode_nbctl lsp-add public public-lr0
> > +check multinode_nbctl lsp-set-type public-lr0 router
> > +check multinode_nbctl lsp-set-addresses public-lr0 router
> > +check multinode_nbctl lsp-set-options public-lr0 router-port=lr0-public
> > +
> > +# localnet port
> > +check multinode_nbctl lsp-add public ln-public
> > +check multinode_nbctl lsp-set-type ln-public localnet
> > +check multinode_nbctl lsp-set-addresses ln-public unknown
> > +check multinode_nbctl lsp-set-options ln-public network_name=public
> > +
> > +check multinode_nbctl lrp-set-gateway-chassis lr0-public ovn-gw-1 20
> > +check multinode_nbctl lr-nat-add lr0 snat 172.16.1.100 10.0.0.0/24
> > +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.16.1.110 10.0.0.3 sw0-port1 50:54:00:00:00:03
> > +check multinode_nbctl lr-nat-add lr0 snat 2000::1 1000::/64
> > +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 2000::2 1000::3 sw0-port1 50:54:00:00:00:03
> > +
> > +check multinode_nbctl --wait=hv sync
> > +
> > +check multinode_nbctl ls-add sw1
> > +check multinode_nbctl lsp-add sw1 sw1-port1
> > +check multinode_nbctl lsp-set-addresses sw1-port1 "40:54:00:00:00:03 20.0.0.3 2000::3"
> > +
> > +check multinode_nbctl lr-add lr1
> > +check multinode_nbctl lrp-add lr1 lr1-sw1 00:00:01:00:ef:01 20.0.0.1/24 2000::a/64
> > +check multinode_nbctl lsp-add sw1 sw1-lr1
> > +check multinode_nbctl lsp-set-type sw1-lr1 router
> > +check multinode_nbctl lsp-set-addresses sw1-lr1 router
> > +check multinode_nbctl lsp-set-options sw1-lr1 router-port=lr1-sw1
> > +
> > +check multinode_nbctl lrp-add lr1 lr1-public 00:00:20:30:22:13 172.16.1.101/24
> > +check multinode_nbctl lsp-add public public-lr1
> > +check multinode_nbctl lsp-set-type public-lr1 router
> > +check multinode_nbctl lsp-set-addresses public-lr1 router
> > +check multinode_nbctl lsp-set-options public-lr1 router-port=lr1-public
> > +
> > +check multinode_nbctl lr-nat-add lr1 snat 172.16.1.101 20.0.0.0/24
> > +check multinode_nbctl lr-nat-add lr1 dnat_and_snat 172.16.1.120 20.0.0.3
> > +check multinode_nbctl lrp-set-gateway-chassis lr1-public ovn-gw-1 20
> > +
> > +check multinode_nbctl --wait=hv sync
> > +
> > +m_as ovn-chassis-1 /data/create_fake_vm.sh sw0-port1 sw0p1 50:54:00:00:00:03 1342 10.0.0.3 24 10.0.0.1 1000::3/64 1000::a
> > +m_as ovn-chassis-2 /data/create_fake_vm.sh sw1-port1 sw1p1 40:54:00:00:00:03 1342 20.0.0.3 24 20.0.0.1 2000::4/64 1000::a
> > +
> > +m_wait_for_ports_up sw0-port1
> > +m_wait_for_ports_up sw1-port1
> > +
> > +m_as ovn-central-az1-1 ovn-sbctl show
> > +
> > +m_as ovn-chassis-1 ovn-appctl  debug/dump-local-datapaths | sort
> > +m_as ovn-chassis-2 ovn-appctl  debug/dump-local-datapaths | sort
> > +
> > +AT_CHECK([m_as ovn-chassis-1 ovn-appctl  debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([m_as ovn-chassis-2 ovn-appctl  debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr1, type: router
> > +Datapath: sw1, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([m_as ovn-gw-1 ovn-appctl  debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: lr1, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Datapath: sw1, type: switch
> > +Local datapaths:
> > +])
> > +
> > +# ping lr0-public IP - 172.168.0.100
> > +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.16.1.100 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +# ping lr1-public IP - 172.168.0.101 from sw0p1
> > +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.16.1.101 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +# ping public ip of sw1-port1 - 172.16.1.120 from sw0p1
> > +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.16.1.120 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +# ping public ip of sw0-port1 - 172.16.1.110 from sw1p1
> > +M_NS_CHECK_EXEC([ovn-chassis-2], [sw1p1], [ping -q -c 3 -i 0.3 -w 2 172.16.1.110 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +# Bind sw0-port2 on chassis-2
> > +m_as ovn-chassis-2 /data/create_fake_vm.sh sw0-port2 sw0p2 50:54:00:00:00:04 1342 10.0.0.4 24 10.0.0.1 1000::4/64 1000::a
> > +m_wait_for_ports_up sw0-port2
> > +
> > +AT_CHECK([m_as ovn-chassis-2 ovn-appctl  debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: lr1, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Datapath: sw1, type: switch
> > +Local datapaths:
> > +])
> > +
> > +# ping public ip of sw0-port1 - 172.16.1.110 from sw0p2
> > +M_NS_CHECK_EXEC([ovn-chassis-2], [sw0p2], [ping -q -c 3 -i 0.3 -w 2 172.16.1.110 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +# ping public ip of sw1-port1 - 172.16.1.120 from sw0p2
> > +M_NS_CHECK_EXEC([ovn-chassis-2], [sw0p2], [ping -q -c 3 -i 0.3 -w 2 172.16.1.120 | FORMAT_PING], \
> > +[0], [dnl
> > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > +])
> > +
> > +# Create a normal router port in public with its peer as a normal distributed router port.
> > +check multinode_nbctl lsp-add public public-lr2
> > +check multinode_nbctl lsp-set-type public-lr2 router
> > +check multinode_nbctl lsp-set-addresses public-lr2 router
> > +check multinode_nbctl lsp-set-options public-lr2 router-port=lr2-public
> > +check multinode_nbctl lr-add lr2
> > +check multinode_nbctl lrp-add lr2 lr2-public 00:00:41:00:1f:61 172.16.1.102/24 3000::a/64
> > +
> > +check multinode_nbctl --wait=hv sync
> > +AT_CHECK([m_as ovn-chassis-1 ovn-appctl  debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: lr1, type: router
> > +Datapath: lr2, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Datapath: sw1, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([m_as ovn-chassis-2 ovn-appctl  debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: lr1, type: router
> > +Datapath: lr2, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Datapath: sw1, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CLEANUP
> > diff --git a/tests/ovn-performance.at b/tests/ovn-performance.at
> > index 7d480c20c8..a003fc36bc 100644
> > --- a/tests/ovn-performance.at
> > +++ b/tests/ovn-performance.at
> > @@ -479,7 +479,7 @@ OVN_CONTROLLER_EXPECT_NO_HIT(
> >  )
> >
> >  OVN_CONTROLLER_EXPECT_HIT_COND(
> > -    [hv1 hv2 hv3 hv4 hv5], [lflow_run], [=0 =0 >0 =0 =0],
> > +    [hv1 hv2 hv3 hv4 hv5], [lflow_run], [>0 >0 >0 =0 =0],
> >      [ovn-nbctl --wait=hv lrp-set-gateway-chassis lr1-public hv3 30 && ovn-nbctl --wait=hv sync]
> >  )
> >
> > @@ -552,8 +552,8 @@ hv5_ch=$(ovn-sbctl --bare --columns _uuid list chassis hv5)
> >  OVS_WAIT_UNTIL([ovn-sbctl find port_binding logical_port=cr-lr1-public chassis=$hv5_ch])
> >  check ovn-nbctl --wait=hv sync
> >  # Delete hv5 from gateway chassis. There should be no lflow_run.
> > -OVN_CONTROLLER_EXPECT_NO_HIT(
> > -    [hv1 hv2 hv3 hv4 hv5], [lflow_run],
> > +OVN_CONTROLLER_EXPECT_HIT_COND(
> > +    [hv1 hv2 hv3 hv4 hv5], [lflow_run], [=0 =0 =0 =0 =0]
> >      [ovn-nbctl --wait=hv lrp-del-gateway-chassis lr1-public hv5]
> >  )
> >
> > diff --git a/tests/ovn.at b/tests/ovn.at
> > index d105ed2535..d4e87212a5 100644
> > --- a/tests/ovn.at
> > +++ b/tests/ovn.at
> > @@ -41481,6 +41481,859 @@ OVN_CLEANUP([hv1])
> >  AT_CLEANUP
> >  ])
> >
> > +OVN_FOR_EACH_NORTHD_NO_HV([
> > +AT_SETUP([ovn-controller -- only_dgp_peer_ports flag in SB datapath_binding])
> > +AT_KEYWORDS([multiple-l3dgw-ports])
> > +ovn_start
> > +net_add n1
> > +sim_add hv1
> > +as hv1
> > +check ovs-vsctl add-br br-phys
> > +ovn_attach n1 br-phys 192.168.0.1
> > +
> > +sim_add hv2
> > +as hv2
> > +check ovs-vsctl add-br br-phys
> > +ovn_attach n1 br-phys 192.168.0.2
> > +
> > +sim_add gw1
> > +as gw1
> > +check ovs-vsctl add-br br-phys
> > +ovn_attach n1 br-phys 192.168.0.3
> > +
> > +sim_add gw2
> > +as gw2
> > +check ovs-vsctl add-br br-phys
> > +ovn_attach n1 br-phys 192.168.0.4
> > +
> > +check ovn-nbctl ls-add sw0
> > +check ovn-nbctl lsp-add sw0 sw0-port1
> > +check ovn-nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:01 10.0.0.3 1000::3"
> > +check ovn-nbctl lsp-add sw0 sw0-port2
> > +check ovn-nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:02 10.0.0.4 1000::4"
> > +
> > +check ovn-nbctl lr-add lr0
> > +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 1000::1/64
> > +check ovn-nbctl lsp-add sw0 sw0-lr0
> > +check ovn-nbctl lsp-set-type sw0-lr0 router
> > +check ovn-nbctl lsp-set-addresses sw0-lr0 router
> > +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> > +
> > +check ovn-nbctl ls-add public
> > +check ovn-nbctl lrp-add lr0 lr0-public 00:00:20:20:12:13 172.168.0.100/24 2000::1/64
> > +check ovn-nbctl lsp-add public public-lr0
> > +check ovn-nbctl lsp-set-type public-lr0 router
> > +check ovn-nbctl lsp-set-addresses public-lr0 router
> > +check ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public
> > +
> > +# localnet port
> > +check ovn-nbctl lsp-add public ln-public
> > +check ovn-nbctl lsp-set-type ln-public localnet
> > +check ovn-nbctl lsp-set-addresses ln-public unknown
> > +check ovn-nbctl lsp-set-options ln-public network_name=phys
> > +
> > +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1 20
> > +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 10.0.0.0/24
> > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 sw0-port2 f0:00:00:01:02:04
> > +check ovn-nbctl lr-nat-add lr0 snat 2000::1 1000::/64
> > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 2000::2 1000::4 sw0-port2 f0:00:00:01:02:04
> > +
> > +check ovn-nbctl --wait=hv sync
> > +
> > +sw0_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=sw0))
> > +lr0_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=lr0))
> > +public_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=public))
> > +
> > +check_offlows_for_datapath() {
> > +    hv=$1
> > +    dp_key=$2
> > +    should_be_present=$3
> > +
> > +    if [[ "$should_be_present" == "yes" ]]; then
> > +        echo "Flows should be present for hv - $hv : datapath - $dp_key"
> > +        OVS_WAIT_UNTIL(
> > +            [test $(as $hv ovs-ofctl dump-flows br-int | grep -c metadata=0x$dp_key) -gt 0]
> > +        )
> > +    else
> > +        echo "Flows should NOT be present for hv - $hv : datapath - $dp_key"
> > +        OVS_WAIT_UNTIL(
> > +            [test $(as $hv ovs-ofctl dump-flows br-int | grep -c metadata=0x$dp_key) -eq 0]
> > +        )
> > +    fi
> > +}
> > +
> > +AT_CHECK([ovn-sbctl get datapath_binding public external_ids:only_dgp_peer_ports], [0], [dnl
> > +"true"
> > +])
> > +
> > +check_offlows_for_datapath hv1 $sw0_dp_key no
> > +check_offlows_for_datapath hv1 $lr0_dp_key no
> > +check_offlows_for_datapath hv1 $public_dp_key no
> > +
> > +check_offlows_for_datapath hv2 $sw0_dp_key no
> > +check_offlows_for_datapath hv2 $lr0_dp_key no
> > +check_offlows_for_datapath hv2 $public_dp_key no
> > +
> > +check_offlows_for_datapath gw1 $sw0_dp_key yes
> > +check_offlows_for_datapath gw1 $lr0_dp_key yes
> > +check_offlows_for_datapath gw1 $public_dp_key yes
> > +
> > +check_offlows_for_datapath gw2 $sw0_dp_key no
> > +check_offlows_for_datapath gw2 $lr0_dp_key no
> > +check_offlows_for_datapath gw2 $public_dp_key no
> > +
> > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> > +Local datapaths:
> > +])
> > +
> > +# Create a VIF on hv1 for sw0-port1
> > +AS_BOX([create a VIF on hv1 for sw0-port1])
> > +
> > +as hv1
> > +ovs-vsctl -- add-port br-int hv1-vif1 -- \
> > +    set interface hv1-vif1 external-ids:iface-id=sw0-port1 \
> > +    options:tx_pcap=hv1/vif1-tx.pcap \
> > +    options:rxq_pcap=hv1/vif1-rx.pcap \
> > +    ofport-request=1
> > +
> > +wait_for_ports_up sw0-port1
> > +
> > +AS_BOX([Create a VIF on hv1 for sw0-port1 - hv1 should have flows for sw0, lr0 and public])
> > +
> > +check_offlows_for_datapath hv1 $sw0_dp_key yes
> > +check_offlows_for_datapath hv1 $lr0_dp_key yes
> > +check_offlows_for_datapath hv1 $public_dp_key yes
> > +
> > +AS_BOX([hv2 should NOT have flows for sw0, lr0 and public])
> > +check_offlows_for_datapath hv2 $sw0_dp_key no
> > +check_offlows_for_datapath hv2 $lr0_dp_key no
> > +check_offlows_for_datapath hv2 $public_dp_key no
> > +
> > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> > +Local datapaths:
> > +])
> > +
> > +AS_BOX([create a switch sw1 and router lr1, attach both and attach lr1 to public])
> > +
> > +check ovn-nbctl ls-add sw1
> > +check ovn-nbctl lsp-add sw1 sw1-port1
> > +check ovn-nbctl lsp-set-addresses sw1-port1 "60:54:00:00:00:01 20.0.0.3"
> > +
> > +check ovn-nbctl lr-add lr1
> > +check ovn-nbctl lrp-add lr1 lr1-sw1 00:00:01:00:ef:01 20.0.0.1/24
> > +check ovn-nbctl lsp-add sw1 sw1-lr1
> > +check ovn-nbctl lsp-set-type sw1-lr1 router
> > +check ovn-nbctl lsp-set-addresses sw1-lr1 router
> > +check ovn-nbctl lsp-set-options sw1-lr1 router-port=lr1-sw1
> > +
> > +check ovn-nbctl lrp-add lr1 lr1-public 00:00:20:30:22:13 172.168.0.101/24
> > +check ovn-nbctl lsp-add public public-lr1
> > +check ovn-nbctl lsp-set-type public-lr1 router
> > +check ovn-nbctl lsp-set-addresses public-lr1 router
> > +check ovn-nbctl lsp-set-options public-lr1 router-port=lr1-public
> > +
> > +sw1_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=sw1))
> > +lr1_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=lr1))
> > +
> > +check ovn-nbctl lr-nat-add lr1 snat 172.168.0.101 20.0.0.0/24
> > +check ovn-nbctl lr-nat-add lr1 dnat_and_snat 172.168.0.140 20.0.0.3
> > +
> > +AS_BOX([create a switch sw2 and router lr2, attach both and attach lr2 to public])
> > +
> > +check ovn-nbctl ls-add sw2
> > +check ovn-nbctl lsp-add sw2 sw2-port1
> > +check ovn-nbctl lsp-set-addresses sw2-port1 "70:54:00:00:00:01 30.0.0.3"
> > +
> > +check ovn-nbctl lr-add lr2
> > +check ovn-nbctl lrp-add lr2 lr2-sw2 00:00:02:00:ef:01 30.0.0.1/24
> > +check ovn-nbctl lsp-add sw2 sw2-lr2
> > +check ovn-nbctl lsp-set-type sw2-lr2 router
> > +check ovn-nbctl lsp-set-addresses sw2-lr2 router
> > +check ovn-nbctl lsp-set-options sw2-lr2 router-port=lr2-sw2
> > +
> > +check ovn-nbctl lrp-add lr2 lr2-public 00:00:20:40:22:53 172.168.0.102/24
> > +check ovn-nbctl lsp-add public public-lr2
> > +check ovn-nbctl lsp-set-type public-lr2 router
> > +check ovn-nbctl lsp-set-addresses public-lr2 router
> > +check ovn-nbctl lsp-set-options public-lr2 router-port=lr2-public
> > +
> > +sw2_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=sw2))
> > +lr2_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=lr2))
> > +
> > +check ovn-nbctl lr-nat-add lr2 snat 172.168.0.102 30.0.0.0/24
> > +check ovn-nbctl lr-nat-add lr2 dnat_and_snat 172.168.0.150 30.0.0.3
> > +
> > +check ovn-nbctl --wait=hv sync
> > +
> > +# Since lr1-public is not a DGP,  public is not a "only_dgp_peer_ports".
> > +AT_CHECK([ovn-sbctl get datapath_binding public external_ids:only_dgp_peer_ports], [1], [ignore], [ignore])
> > +
> > +check_offlows_for_datapath hv1 $sw1_dp_key yes
> > +check_offlows_for_datapath hv1 $lr1_dp_key yes
> > +check_offlows_for_datapath hv1 $sw2_dp_key yes
> > +check_offlows_for_datapath hv1 $lr2_dp_key yes
> > +
> > +check_offlows_for_datapath hv2 $sw1_dp_key no
> > +check_offlows_for_datapath hv2 $lr1_dp_key no
> > +check_offlows_for_datapath hv2 $sw2_dp_key no
> > +check_offlows_for_datapath hv2 $lr2_dp_key no
> > +
> > +check_offlows_for_datapath gw1 $sw1_dp_key yes
> > +check_offlows_for_datapath gw1 $lr1_dp_key yes
> > +check_offlows_for_datapath gw1 $sw2_dp_key yes
> > +check_offlows_for_datapath gw1 $lr2_dp_key yes
> > +
> > +check_offlows_for_datapath gw2 $sw1_dp_key no
> > +check_offlows_for_datapath gw2 $lr1_dp_key no
> > +check_offlows_for_datapath gw2 $sw2_dp_key no
> > +check_offlows_for_datapath gw2 $lr2_dp_key no
> > +
> > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: lr1, type: router
> > +Datapath: lr2, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Datapath: sw1, type: switch
> > +Datapath: sw2, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: lr1, type: router
> > +Datapath: lr2, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Datapath: sw1, type: switch
> > +Datapath: sw2, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> > +Local datapaths:
> > +])
> > +
> > +AS_BOX([Set gw2 as gateway chassis for lr1-public and lr2-public])
> > +check ovn-nbctl --wait=hv lrp-set-gateway-chassis lr1-public gw2 20
> > +check ovn-nbctl --wait=hv lrp-set-gateway-chassis lr2-public gw2 30
> > +wait_row_count Port_Binding 1 logical_port=cr-lr1-public
> > +wait_row_count Port_Binding 1 logical_port=cr-lr2-public
> > +
> > +AT_CHECK([ovn-sbctl get datapath_binding public external_ids:only_dgp_peer_ports], [0], [dnl
> > +"true"
> > +])
> > +
> > +check ovn-nbctl --wait=hv sync
> > +
> > +check_offlows_for_datapath hv1 $sw0_dp_key yes
> > +check_offlows_for_datapath hv1 $lr0_dp_key yes
> > +check_offlows_for_datapath hv1 $public_dp_key yes
> > +check_offlows_for_datapath hv1 $sw1_dp_key no
> > +check_offlows_for_datapath hv1 $lr1_dp_key no
> > +check_offlows_for_datapath hv1 $sw2_dp_key no
> > +check_offlows_for_datapath hv1 $lr2_dp_key no
> > +
> > +check_offlows_for_datapath hv2 $sw0_dp_key no
> > +check_offlows_for_datapath hv2 $lr0_dp_key no
> > +check_offlows_for_datapath hv2 $public_dp_key no
> > +check_offlows_for_datapath hv2 $sw1_dp_key no
> > +check_offlows_for_datapath hv2 $lr1_dp_key no
> > +check_offlows_for_datapath hv2 $sw2_dp_key no
> > +check_offlows_for_datapath hv2 $lr2_dp_key no
> > +
> > +check_offlows_for_datapath gw1 $sw0_dp_key yes
> > +check_offlows_for_datapath gw1 $lr0_dp_key yes
> > +check_offlows_for_datapath gw1 $public_dp_key yes
> > +check_offlows_for_datapath gw1 $sw1_dp_key no
> > +check_offlows_for_datapath gw1 $lr1_dp_key no
> > +check_offlows_for_datapath gw1 $sw2_dp_key no
> > +check_offlows_for_datapath gw1 $lr2_dp_key no
> > +
> > +check_offlows_for_datapath gw2 $sw0_dp_key no
> > +check_offlows_for_datapath gw2 $lr0_dp_key no
> > +check_offlows_for_datapath hv1 $public_dp_key yes
> > +check_offlows_for_datapath gw2 $sw1_dp_key yes
> > +check_offlows_for_datapath gw2 $lr1_dp_key yes
> > +check_offlows_for_datapath gw2 $sw2_dp_key yes
> > +check_offlows_for_datapath gw2 $lr2_dp_key yes
> > +
> > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr1, type: router
> > +Datapath: lr2, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw1, type: switch
> > +Datapath: sw2, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AS_BOX([Create a VIF on hv2 for sw1-port1])
> > +
> > +as hv2
> > +ovs-vsctl -- add-port br-int hv2-vif1 -- \
> > +    set interface hv2-vif1 external-ids:iface-id=sw1-port1 \
> > +    options:tx_pcap=hv2/vif1-tx.pcap \
> > +    options:rxq_pcap=hv2/vif1-rx.pcap \
> > +    ofport-request=1
> > +
> > +wait_for_ports_up sw1-port1
> > +
> > +check_offlows_for_datapath hv1 $sw0_dp_key yes
> > +check_offlows_for_datapath hv1 $lr0_dp_key yes
> > +check_offlows_for_datapath hv1 $public_dp_key yes
> > +check_offlows_for_datapath hv1 $sw1_dp_key no
> > +check_offlows_for_datapath hv1 $lr1_dp_key no
> > +check_offlows_for_datapath hv1 $sw2_dp_key no
> > +check_offlows_for_datapath hv1 $lr2_dp_key no
> > +
> > +check_offlows_for_datapath hv2 $sw0_dp_key no
> > +check_offlows_for_datapath hv2 $lr0_dp_key no
> > +
> > +# Since there are no distributed dnat_and_snat entries
> > +# in lr1, hv2 will not have "public" in its
> > +# local datapaths.
> > +check_offlows_for_datapath hv2 $public_dp_key no
> > +check_offlows_for_datapath hv2 $sw1_dp_key yes
> > +check_offlows_for_datapath hv2 $lr1_dp_key yes
> > +check_offlows_for_datapath hv2 $sw2_dp_key no
> > +check_offlows_for_datapath hv2 $lr2_dp_key no
> > +
> > +check_offlows_for_datapath gw1 $sw0_dp_key yes
> > +check_offlows_for_datapath gw1 $lr0_dp_key yes
> > +check_offlows_for_datapath gw1 $public_dp_key yes
> > +check_offlows_for_datapath gw1 $sw1_dp_key no
> > +check_offlows_for_datapath gw1 $lr1_dp_key no
> > +check_offlows_for_datapath gw1 $sw2_dp_key no
> > +check_offlows_for_datapath gw1 $lr2_dp_key no
> > +
> > +# gw2 should have sw1, lr1, sw2 and lr2 and public in its local datapaths.
> > +check_offlows_for_datapath gw2 $sw0_dp_key no
> > +check_offlows_for_datapath gw2 $lr0_dp_key no
> > +check_offlows_for_datapath gw2 $public_dp_key yes
> > +check_offlows_for_datapath gw2 $sw1_dp_key yes
> > +check_offlows_for_datapath gw2 $lr1_dp_key yes
> > +check_offlows_for_datapath gw2 $sw2_dp_key yes
> > +check_offlows_for_datapath gw2 $lr2_dp_key yes
> > +
> > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr1, type: router
> > +Datapath: sw1, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr1, type: router
> > +Datapath: lr2, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw1, type: switch
> > +Datapath: sw2, type: switch
> > +Local datapaths:
> > +])
> > +
> > +# Add distributed dnat_and_snat in lr1.  hv2 should have
> > +# public in its local datapaths.
> > +AS_BOX([ Add distributed dnat_and_snat in lr1])
> > +
> > +check ovn-nbctl lr-nat-del lr1 dnat_and_snat
> > +check ovn-nbctl --wait=hv lr-nat-add lr1 dnat_and_snat 172.168.0.140 20.0.0.3 sw1-port1 10:00:00:01:02:14
> > +
> > +check_offlows_for_datapath hv2 $sw0_dp_key no
> > +check_offlows_for_datapath hv2 $lr0_dp_key no
> > +check_offlows_for_datapath hv2 $public_dp_key yes
> > +check_offlows_for_datapath hv2 $sw1_dp_key yes
> > +check_offlows_for_datapath hv2 $lr1_dp_key yes
> > +check_offlows_for_datapath hv2 $sw2_dp_key no
> > +check_offlows_for_datapath hv2 $lr2_dp_key no
> > +
> > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr1, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw1, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr1, type: router
> > +Datapath: lr2, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw1, type: switch
> > +Datapath: sw2, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AS_BOX([Create a VIF on hv2 for sw0-port2])
> > +
> > +as hv2
> > +ovs-vsctl -- add-port br-int hv2-vif2 -- \
> > +    set interface hv2-vif2 external-ids:iface-id=sw0-port2 \
> > +    options:tx_pcap=hv2/vif2-tx.pcap \
> > +    options:rxq_pcap=hv2/vif2-rx.pcap \
> > +    ofport-request=2
> > +
> > +wait_for_ports_up sw0-port2
> > +
> > +check_offlows_for_datapath hv2 $sw0_dp_key yes
> > +check_offlows_for_datapath hv2 $lr0_dp_key yes
> > +check_offlows_for_datapath hv2 $public_dp_key yes
> > +check_offlows_for_datapath hv2 $sw1_dp_key yes
> > +check_offlows_for_datapath hv2 $lr1_dp_key yes
> > +check_offlows_for_datapath hv2 $sw2_dp_key no
> > +check_offlows_for_datapath hv2 $lr2_dp_key no
> > +
> > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: lr1, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Datapath: sw1, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr1, type: router
> > +Datapath: lr2, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw1, type: switch
> > +Datapath: sw2, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AS_BOX([Delete the VIF for sw1-port1 in hv2])
> > +
> > +as hv2 ovs-vsctl del-port hv2-vif1
> > +check ovn-nbctl --wait=hv sync
> > +check_column "false" Port_Binding up logical_port=sw1-port1
> > +
> > +check_offlows_for_datapath hv2 $sw0_dp_key yes
> > +check_offlows_for_datapath hv2 $lr0_dp_key yes
> > +check_offlows_for_datapath hv2 $public_dp_key yes
> > +check_offlows_for_datapath hv2 $sw1_dp_key no
> > +check_offlows_for_datapath hv2 $lr1_dp_key no
> > +check_offlows_for_datapath hv2 $sw2_dp_key no
> > +check_offlows_for_datapath hv2 $lr2_dp_key no
> > +
> > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr1, type: router
> > +Datapath: lr2, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw1, type: switch
> > +Datapath: sw2, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AS_BOX([Delete the VIF for sw0-port2 in hv2])
> > +
> > +# Presently when a port binding is released we are not
> > +# deleting its datapath from the local_datapaths if it
> > +# is not relevant anymore.
> > +
> > +as hv2 ovs-vsctl del-port hv2-vif2
> > +check ovn-nbctl --wait=hv sync
> > +check_column "false" Port_Binding up logical_port=sw0-port2
> > +
> > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +# hv2 would still have public, sw0 and lr0 in its local datapaths.
> > +# Next recompute should delete these datapaths.
> > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr1, type: router
> > +Datapath: lr2, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw1, type: switch
> > +Datapath: sw2, type: switch
> > +Local datapaths:
> > +])
> > +
> > +# Trigger a recompute
> > +AS_BOX([Trigger a recompute in hv2])
> > +check as hv2 ovn-appctl inc-engine/recompute
> > +
> > +check_offlows_for_datapath hv2 $sw0_dp_key no
> > +check_offlows_for_datapath hv2 $lr0_dp_key no
> > +check_offlows_for_datapath hv2 $public_dp_key no
> > +check_offlows_for_datapath hv2 $sw1_dp_key no
> > +check_offlows_for_datapath hv2 $lr1_dp_key no
> > +check_offlows_for_datapath hv2 $sw2_dp_key no
> > +check_offlows_for_datapath hv2 $lr2_dp_key no
> > +
> > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr1, type: router
> > +Datapath: lr2, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw1, type: switch
> > +Datapath: sw2, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AS_BOX([Disconnect sw2 from lr2])
> > +
> > +check ovn-nbctl --wait=hv lsp-set-options sw2-lr2 router-port=lr2-sw2xxx
> > +check_offlows_for_datapath hv1 $sw0_dp_key yes
> > +check_offlows_for_datapath hv1 $lr0_dp_key yes
> > +check_offlows_for_datapath hv1 $public_dp_key yes
> > +check_offlows_for_datapath hv1 $sw1_dp_key no
> > +check_offlows_for_datapath hv1 $lr1_dp_key no
> > +check_offlows_for_datapath hv1 $sw2_dp_key no
> > +check_offlows_for_datapath hv1 $lr2_dp_key no
> > +
> > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr1, type: router
> > +Datapath: lr2, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw1, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AS_BOX([Reconnect sw2 to lr2 again])
> > +
> > +check ovn-nbctl --wait=hv lsp-set-options sw2-lr2 router-port=lr2-sw2
> > +check_offlows_for_datapath hv1 $sw0_dp_key yes
> > +check_offlows_for_datapath hv1 $lr0_dp_key yes
> > +check_offlows_for_datapath hv1 $public_dp_key yes
> > +check_offlows_for_datapath hv1 $sw1_dp_key no
> > +check_offlows_for_datapath hv1 $lr1_dp_key no
> > +check_offlows_for_datapath hv1 $sw2_dp_key no
> > +check_offlows_for_datapath hv1 $lr2_dp_key no
> > +
> > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr1, type: router
> > +Datapath: lr2, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw1, type: switch
> > +Datapath: sw2, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AS_BOX([Create a VIF on gw2 for sw1-port1])
> > +
> > +as gw2
> > +ovs-vsctl -- add-port br-int gw2-vif2 -- \
> > +    set interface gw2-vif2 external-ids:iface-id=sw1-port1 \
> > +    options:tx_pcap=gw2/vif2-tx.pcap \
> > +    options:rxq_pcap=gw2/vif2-rx.pcap \
> > +    ofport-request=2
> > +
> > +wait_for_ports_up sw1-port1
> > +
> > +check_offlows_for_datapath gw2 $sw0_dp_key no
> > +check_offlows_for_datapath gw2 $lr0_dp_key no
> > +check_offlows_for_datapath gw2 $public_dp_key yes
> > +check_offlows_for_datapath gw2 $sw1_dp_key yes
> > +check_offlows_for_datapath gw2 $lr1_dp_key yes
> > +check_offlows_for_datapath gw2 $sw2_dp_key yes
> > +check_offlows_for_datapath gw2 $lr2_dp_key yes
> > +
> > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr1, type: router
> > +Datapath: lr2, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw1, type: switch
> > +Datapath: sw2, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AS_BOX([Delete the VIF for sw1-port1 in gw2])
> > +
> > +as gw2 ovs-vsctl del-port gw2-vif2
> > +check ovn-nbctl --wait=hv sync
> > +check_column "false" Port_Binding up logical_port=sw1-port1
> > +
> > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +# hv2 would still have public in its local datapaths.  Next recompute should
> > +# delete this datapath from the local datapaths.
> > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr1, type: router
> > +Datapath: lr2, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw1, type: switch
> > +Datapath: sw2, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AS_BOX([Create a logical port for public and bind it on hv2])
> > +# hv2 will only have public in its local datapaths.
> > +check ovn-nbctl lsp-add public public-p1
> > +
> > +as hv2
> > +ovs-vsctl -- add-port br-int hv2-vif3 -- \
> > +    set interface hv2-vif3 external-ids:iface-id=public-p1 \
> > +    options:tx_pcap=hv2/vif3-tx.pcap \
> > +    options:rxq_pcap=hv2/vif3-rx.pcap \
> > +    ofport-request=2
> > +
> > +wait_for_ports_up public-p1
> > +
> > +# as hv2 ovn-appctl -t ovn-controller inc-engine/recompute
> > +# check ovn-nbctl --wait=hv sync
> > +
> > +as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort
> > +
> > +check_offlows_for_datapath hv2 $sw0_dp_key no
> > +check_offlows_for_datapath hv2 $lr0_dp_key no
> > +check_offlows_for_datapath hv2 $public_dp_key yes
> > +check_offlows_for_datapath hv2 $sw1_dp_key no
> > +check_offlows_for_datapath hv2 $lr1_dp_key no
> > +check_offlows_for_datapath hv2 $sw2_dp_key no
> > +check_offlows_for_datapath hv2 $lr2_dp_key no
> > +
> > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: public, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr0, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw0, type: switch
> > +Local datapaths:
> > +])
> > +
> > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > +Datapath: lr1, type: router
> > +Datapath: lr2, type: router
> > +Datapath: public, type: switch
> > +Datapath: sw1, type: switch
> > +Datapath: sw2, type: switch
> > +Local datapaths:
> > +])
> > +
> > +OVN_CLEANUP([hv1], [hv2], [gw1], [gw2])
> > +AT_CLEANUP
> > +])
> > +
> >  OVN_FOR_EACH_NORTHD([
> >  AT_SETUP([requested-tnl-key-recompute])
> >  AT_KEYWORDS([requested-tnl-key-recompute])
> > --
> > 2.48.1
> >
> > _______________________________________________
> > dev mailing list
> > dev@openvswitch.org
> > https://mail.openvswitch.org/mailman/listinfo/ovs-dev

On Tue, Feb 11, 2025 at 01:21:17PM -0500, Numan Siddique wrote:
> On Tue, Feb 11, 2025 at 5:15 AM Felix Huettner
> <felix.huettner@stackit.cloud> wrote:
> >
> > On Mon, Feb 10, 2025 at 10:37:25AM -0500, numans@ovn.org wrote:
> > > From: Numan Siddique <numans@ovn.org>
> > >
> > > Consider the below logical topology
> > >
> > > sw0-p1 -
> > >         |
> > > sw0-p2 -   ->  sw0 -> lr0 ----
> > > ...     |                     |
> > > sw0-pn -                      |
> > >                               |
> > > sw1-p1 -                      |
> > >         |                     |
> > > sw1p-2 -   ->  sw1 -> lr1 ----  --- public (provider switch)
> > > ...     |                     |
> > > sw1-pn-                       |
> > >                               |
> > > swn-p1 -                      |
> > >         |                     |
> > > swn-p2-    ->  swn -> lrn ----
> > > ...     |
> > > swn-pn -
> > >
> > > All the routers are connected to the provider switch via
> > > a ditributed gateway port.
> > >
> > > If sw0-p1 is resident on the chassis C1, then since there is a path
> > > to all the switches and the routers, ovn-controller will add all
> > > these datapaths to its 'local_datapaths' map.  This in turn results
> > > in processing all the logical flows and installing all the openflows
> > > and in turn wasting the CPU time.  This can be very costly in
> > > a highly scaled deployment.
> > >
> > > Previous commit sets a flag "only_dgp_peer_ports" in the SB Datapath
> > > binding for a provider switch (with only dgp peer ports).
> > >
> > > In this commit, ovn-controller makes use of this flag and stops
> > > adding other datapaths connected to the public provider switch
> > > to the 'local_datapaths'.
> > >
> > > For example, when it claims sw0-p1, it adds sw0, lr0 and public
> > > to the local_datapaths and stops there.  If it later claims
> > > sw1-p1, it will add sw1 and lr1.
> > >
> > > This reduces the recompute time and the number of openflow rules
> > > added to ovs-vswitchd significantly.
> > >
> > > I tested this patch with a deployment of below logical resources:
> > >
> > > No of logical switches - 778
> > > No of logical routers  - 871
> > > No of logical flows    - 85626
> > > No of 'ovn-sbctl dump-flows' - 208631
> > >
> > > Without this patch, afte claiming sw0-p1, ovn-controller adds
> > > 269098 openflow rules and it takes approx 2500 milli seconds
> > > for a recompute.
> > >
> > > With this patch, after claiming sw0-p1, ovn-controller adds
> > > 21350 openflow rules and it takes approx 280 milli seconds
> > > for a recompute.
> > >
> > > There is approx 90% reduction in the openflow rules and
> > > 88% reduction in recompute time when a comoute node has
> > > VIFs from one logical switch.
> >
> > Hi Numan,
> >
> > that sounds really great. We have a similar setup that would heavily
> > benefit from this.
> >
> > However when reading through the patch i have one concern about the
> > correctness of not considering these ports, but that might just come
> > from misunderstanding something.
> >
> > I assumed that we have one chassis c1 that has claimed sw0-p1. It should
> > then know sw0, lr0 and public.
> > I also assumed that there is chassis c2 claiming sw1-p1. That would then
> > know sw1, lr1 and public.
> > Then i assumed there is a chassis g1 that has claimed the lr0 CR port.
> > Also chassis g2 has claimed the lr1 CR port.
> >
> > Now we add a NAT rule to lr0. It is configured with dnat_and_snat and
> > the settings necessary to process it in a distributed way (i think by
> > setting logical_port and stateless, but i may be wrong here).
> 
> It's actually logical_port and external_mac
> 
> It maps
> > the sw0-p1 IP to some IP on public.
> >
> > On lr1 we have a similar setup of NAT entries for the sw1-p1 IP.
> >
> > If i understand these NAT settins correctly this should now lead to traffic
> > from sw0-p1 to sw1-p1 to flow directly (when using the external ip of the
> > sw1-p1 NAT rule). So the traffic would flow from chassic c1 to c2
> > without any involvement of g1 and g2.
> >
> That's correct.
> 
> > In this case wouldn't chassis c1 need to know about the lr1 and sw1
> > datapath?
> 
> Not really.  Packet from sw0-p1 on C1 will enter lr0 pipeline first.
> If there is a mac_binding entry
> for the public IP of sw1-p1 in the lr0 datapath, it will set the
> eth.dst to this mac and sends
> the packet to public switch datapath and then to the wire via the localnet port.
> The fabric will make sure that the packet reaches C2.
> If there is no mac_binding entry, ovn-controller will generate an ARP
> request for the sw1-p1 public IP.
> And then eventually reinjects the packet after it learns the mac.

Hi Numan,

thanks a lot for the explanation.
I always had the misunderstanding that such traffic would go via the
overlay instead of via the localnet port.
With the localnet port that makes a lot more sense.

> 
> The same happens on the C2 for the reply path.
> 
> Even without this patch,  I think this is the behavior.
> 
> There is a multinode system test to cover this scenario.
> 
> Hope this addresses your concern.  Let me know if you have further questions.

Yes this was really helpful.

Thanks a lot,
Felix

> 
> Numan
> 
> >
> > I guess i missed something somewhere, but since i am not sure i wanted
> > to share this concern.
> >
> > Thanks a lot,
> > Felix
> >
> > >
> > > Signed-off-by: Numan Siddique <numans@ovn.org>
> > > ---
> > >  controller/binding.c        | 249 +++++++++--
> > >  controller/binding.h        |   2 +
> > >  controller/local_data.c     |  84 +++-
> > >  controller/local_data.h     |   6 +
> > >  controller/lport.c          |  12 +
> > >  controller/lport.h          |   4 +
> > >  controller/ovn-controller.c |  38 ++
> > >  tests/multinode.at          | 178 ++++++++
> > >  tests/ovn-performance.at    |   6 +-
> > >  tests/ovn.at                | 853 ++++++++++++++++++++++++++++++++++++
> > >  10 files changed, 1394 insertions(+), 38 deletions(-)
> > >
> > > diff --git a/controller/binding.c b/controller/binding.c
> > > index ea5bf5a9fd..88c9640c3a 100644
> > > --- a/controller/binding.c
> > > +++ b/controller/binding.c
> > > @@ -825,6 +825,16 @@ static bool binding_lport_update_port_sec(
> > >  static bool ovs_iface_matches_lport_iface_id_ver(
> > >      const struct ovsrec_interface *,
> > >      const struct sbrec_port_binding *);
> > > +static bool cleanup_patch_port_local_dps(
> > > +    const struct sbrec_port_binding *, const struct sbrec_port_binding *cr_pb,
> > > +    const struct sbrec_port_binding *peer, struct local_datapath *ld,
> > > +    struct binding_ctx_in *b_ctx_in,
> > > +    struct binding_ctx_out *b_ctx_out,
> > > +    bool *cleanup);
> > > +static bool local_datapath_is_relevant(
> > > +    struct local_datapath *, struct local_datapath *ignore_peer_ld,
> > > +    struct hmap *local_datapaths, int *depth, const struct sbrec_chassis *,
> > > +    struct ovsdb_idl_index *);
> > >
> > >  void
> > >  related_lports_init(struct related_lports *rp)
> > > @@ -1062,6 +1072,13 @@ binding_dump_related_lports(struct related_lports *related_lports,
> > >      }
> > >  }
> > >
> > > +struct dp_binding {
> > > +    struct hmap_node key_node;
> > > +
> > > +    uint32_t dp_key;
> > > +    struct hmapx binding_lports;
> > > +};
> > > +
> > >  void
> > >  binding_dump_local_bindings(struct local_binding_data *lbinding_data,
> > >                              struct ds *out_data)
> > > @@ -1130,6 +1147,19 @@ binding_dump_local_bindings(struct local_binding_data *lbinding_data,
> > >      free(nodes);
> > >  }
> > >
> > > +void
> > > +binding_dump_local_datapaths(struct hmap *local_datapaths,
> > > +                             struct ds *out_data)
> > > +{
> > > +    ds_put_cstr(out_data, "Local datapaths:\n");
> > > +    struct local_datapath *ld;
> > > +    HMAP_FOR_EACH (ld, hmap_node, local_datapaths) {
> > > +        ds_put_format(out_data, "Datapath: %s, type: %s\n",
> > > +                      smap_get(&ld->datapath->external_ids, "name"),
> > > +                      ld->is_switch ? "switch" : "router");
> > > +    }
> > > +}
> > > +
> > >  void
> > >  set_pb_chassis_in_sbrec(const struct sbrec_port_binding *pb,
> > >                          const struct sbrec_chassis *chassis_rec,
> > > @@ -2137,7 +2167,9 @@ build_local_bindings(struct binding_ctx_in *b_ctx_in,
> > >
> > >  static bool consider_patch_port_for_local_datapaths(
> > >          const struct sbrec_port_binding *,
> > > -        struct binding_ctx_in *, struct binding_ctx_out *);
> > > +        const struct sbrec_port_binding *cr_pb,
> > > +        struct binding_ctx_in *, struct binding_ctx_out *,
> > > +        bool check_and_remove_localdps);
> > >
> > >  void
> > >  binding_run(struct binding_ctx_in *b_ctx_in, struct binding_ctx_out *b_ctx_out)
> > > @@ -2182,7 +2214,8 @@ binding_run(struct binding_ctx_in *b_ctx_in, struct binding_ctx_out *b_ctx_out)
> > >          switch (lport_type) {
> > >          case LP_PATCH:
> > >              update_related_lport(pb, b_ctx_out);
> > > -            consider_patch_port_for_local_datapaths(pb, b_ctx_in, b_ctx_out);
> > > +            consider_patch_port_for_local_datapaths(pb, NULL, b_ctx_in,
> > > +                                                    b_ctx_out, false);
> > >              break;
> > >
> > >          case LP_VTEP:
> > > @@ -2238,6 +2271,9 @@ binding_run(struct binding_ctx_in *b_ctx_in, struct binding_ctx_out *b_ctx_out)
> > >              struct lport *lnet_lport = xmalloc(sizeof *lnet_lport);
> > >              lnet_lport->pb = pb;
> > >              ovs_list_push_back(&localnet_lports, &lnet_lport->list_node);
> > > +            if (pb->chassis == b_ctx_in->chassis_rec) {
> > > +                sbrec_port_binding_set_chassis(pb, NULL);
> > > +            }
> > >              break;
> > >          }
> > >
> > > @@ -2566,7 +2602,6 @@ consider_iface_release(const struct ovsrec_interface *iface_rec,
> > >                                             lbinding->iface->name,
> > >                                             &lbinding->iface->header_.uuid);
> > >              }
> > > -
> > >          } else if (b_lport && b_lport->type == LP_LOCALPORT) {
> > >              /* lbinding is associated with a localport.  Remove it from the
> > >               * related lports. */
> > > @@ -2956,12 +2991,27 @@ handle_updated_vif_lport(const struct sbrec_port_binding *pb,
> > >
> > >  static bool
> > >  consider_patch_port_for_local_datapaths(const struct sbrec_port_binding *pb,
> > > +                                        const struct sbrec_port_binding *cr_pb,
> > >                                          struct binding_ctx_in *b_ctx_in,
> > > -                                        struct binding_ctx_out *b_ctx_out)
> > > +                                        struct binding_ctx_out *b_ctx_out,
> > > +                                        bool check_and_remove_localdps)
> > >  {
> > > -    struct local_datapath *ld =
> > > -        get_local_datapath(b_ctx_out->local_datapaths,
> > > -                           pb->datapath->tunnel_key);
> > > +    const struct sbrec_port_binding *peer;
> > > +    struct local_datapath *peer_ld = NULL;
> > > +    struct local_datapath *ld = NULL;
> > > +
> > > +    ld = get_local_datapath(b_ctx_out->local_datapaths,
> > > +                            pb->datapath->tunnel_key);
> > > +    if (ld && ld->has_only_dgp_peer_ports) {
> > > +        /* Nothing much to do. */
> > > +        return true;
> > > +    }
> > > +
> > > +    peer = lport_get_peer(pb, b_ctx_in->sbrec_port_binding_by_name);
> > > +    if (peer) {
> > > +        peer_ld = get_local_datapath(b_ctx_out->local_datapaths,
> > > +                                     peer->datapath->tunnel_key);
> > > +    }
> > >
> > >      if (!ld) {
> > >          /* If 'ld' for this lport is not present, then check if
> > > @@ -2969,17 +3019,9 @@ consider_patch_port_for_local_datapaths(const struct sbrec_port_binding *pb,
> > >           * and peer's datapath is already in the local datapaths,
> > >           * then add this lport's datapath to the local_datapaths.
> > >           * */
> > > -        const struct sbrec_port_binding *peer;
> > > -        struct local_datapath *peer_ld = NULL;
> > > -        peer = lport_get_peer(pb, b_ctx_in->sbrec_port_binding_by_name);
> > > -        if (peer) {
> > > -            peer_ld =
> > > -                get_local_datapath(b_ctx_out->local_datapaths,
> > > -                                   peer->datapath->tunnel_key);
> > > -        }
> > > -        if (peer_ld && need_add_peer_to_local(
> > > -                b_ctx_in->sbrec_port_binding_by_name, peer,
> > > -                b_ctx_in->chassis_rec)) {
> > > +        if (peer_ld && !peer_ld->has_only_dgp_peer_ports &&
> > > +            need_add_peer_to_local(b_ctx_in->sbrec_port_binding_by_name, peer,
> > > +                                   b_ctx_in->chassis_rec)) {
> > >              ld = add_local_datapath(
> > >                      b_ctx_in->sbrec_datapath_binding_by_key,
> > >                      b_ctx_in->sbrec_port_binding_by_datapath,
> > > @@ -2992,7 +3034,7 @@ consider_patch_port_for_local_datapaths(const struct sbrec_port_binding *pb,
> > >          /* Add the peer datapath to the local datapaths if it's
> > >           * not present yet.
> > >           */
> > > -        if (need_add_peer_to_local(
> > > +        if (peer && need_add_peer_to_local(
> > >                  b_ctx_in->sbrec_port_binding_by_name, pb,
> > >                  b_ctx_in->chassis_rec)) {
> > >              add_local_datapath_peer_port(
> > > @@ -3003,6 +3045,18 @@ consider_patch_port_for_local_datapaths(const struct sbrec_port_binding *pb,
> > >                  ld, b_ctx_out->local_datapaths,
> > >                  b_ctx_out->tracked_dp_bindings);
> > >          }
> > > +
> > > +        if (check_and_remove_localdps) {
> > > +            bool cleanedup = false;
> > > +            if (!cleanup_patch_port_local_dps(pb, cr_pb, peer, ld, b_ctx_in,
> > > +                                              b_ctx_out, &cleanedup)) {
> > > +                return false;
> > > +            }
> > > +
> > > +            if (cleanedup) {
> > > +                ld = NULL;
> > > +            }
> > > +        }
> > >      }
> > >
> > >      /* If this chassis is requested - try to claim. */
> > > @@ -3021,12 +3075,10 @@ consider_patch_port_for_local_datapaths(const struct sbrec_port_binding *pb,
> > >          || if_status_is_port_claimed(b_ctx_out->if_mgr, pb->logical_port)) {
> > >
> > >          remove_local_lports(pb->logical_port, b_ctx_out);
> > > -        if (!release_lport(pb, ld, b_ctx_in->chassis_rec,
> > > -                           !b_ctx_in->ovnsb_idl_txn,
> > > -                           b_ctx_out->tracked_dp_bindings,
> > > -                           b_ctx_out->if_mgr)) {
> > > -            return false;
> > > -        }
> > > +        return release_lport(pb, ld, b_ctx_in->chassis_rec,
> > > +                             !b_ctx_in->ovnsb_idl_txn,
> > > +                             b_ctx_out->tracked_dp_bindings,
> > > +                             b_ctx_out->if_mgr);
> > >      }
> > >      return true;
> > >  }
> > > @@ -3086,8 +3138,8 @@ handle_updated_port(struct binding_ctx_in *b_ctx_in,
> > >
> > >      case LP_PATCH:
> > >          update_related_lport(pb, b_ctx_out);
> > > -        handled = consider_patch_port_for_local_datapaths(pb, b_ctx_in,
> > > -                                                              b_ctx_out);
> > > +        handled = consider_patch_port_for_local_datapaths(pb, NULL, b_ctx_in,
> > > +                                                          b_ctx_out, true);
> > >          break;
> > >
> > >      case LP_VTEP:
> > > @@ -3130,8 +3182,8 @@ handle_updated_port(struct binding_ctx_in *b_ctx_in,
> > >              break;
> > >          }
> > >          handled = consider_patch_port_for_local_datapaths(distributed_pb,
> > > -                                                          b_ctx_in,
> > > -                                                          b_ctx_out);
> > > +                                                          pb, b_ctx_in,
> > > +                                                          b_ctx_out, true);
> > >          break;
> > >
> > >      case LP_EXTERNAL:
> > > @@ -3901,3 +3953,142 @@ binding_destroy(void)
> > >      shash_destroy_free_data(&_qos_ports);
> > >      sset_clear(&_postponed_ports);
> > >  }
> > > +
> > > +static bool
> > > +is_patch_pb_chassis_relevant(
> > > +    const struct sbrec_port_binding *pb,
> > > +    const struct sbrec_chassis *chassis,
> > > +    struct ovsdb_idl_index *sbrec_port_binding_by_name)
> > > +{
> > > +    if (ha_chassis_group_contains(pb->ha_chassis_group, chassis)) {
> > > +        return true;
> > > +    }
> > > +
> > > +    const struct sbrec_port_binding *pb_crp =
> > > +        lport_get_cr_port(sbrec_port_binding_by_name, pb);
> > > +    if (pb_crp) {
> > > +        return ha_chassis_group_contains(pb_crp->ha_chassis_group, chassis);
> > > +    }
> > > +
> > > +    return false;
> > > +}
> > > +
> > > +static bool
> > > +cleanup_patch_port_local_dps(const struct sbrec_port_binding *pb,
> > > +                             const struct sbrec_port_binding *cr_pb,
> > > +                             const struct sbrec_port_binding *peer,
> > > +                             struct local_datapath *ld,
> > > +                             struct binding_ctx_in *b_ctx_in,
> > > +                             struct binding_ctx_out *b_ctx_out,
> > > +                             bool *cleanedup)
> > > +{
> > > +    *cleanedup = false;
> > > +    if (!peer) {
> > > +        /* Remove 'pb' from the ld's peer ports as it has no peer. */
> > > +        remove_local_datapath_peer_port(pb, ld,
> > > +                                        b_ctx_out->local_datapaths);
> > > +    }
> > > +
> > > +    /* We can consider removing the 'ld' of the patch port 'pb' from the
> > > +     * local datapaths, if all the below conditions are met
> > > +     *     - 'pb' doesn't have a peer or ld' is a router datapath
> > > +     *     - if 'pb' is a distributed gateway port (dgp), then
> > > +     *       its chassisredirect port's ha chassis group doesn't
> > > +     *       contain our 'chassis rec'
> > > +     *     - and finally 'ld' is not relevant any more.  See
> > > +     *       local_datapath_is_relevant() for more details.
> > > +     *
> > > +     * Note: If 'ld' can be removed, then all its connected local datapaths
> > > +     * can also be removed.
> > > +     *
> > > +     * For example, if we had sw1-port1 ->  sw1 -> lr1 -> sw2 and if
> > > +     * sw1-port1 resides on this chassis, and if the link between sw1 and
> > > +     * lr1 is broken, then we can remove lr1 and sw2 from the
> > > +     * local_datapaths.
> > > +     * */
> > > +
> > > +    bool consider_ld_for_removal = !peer || !ld->is_switch;
> > > +    if (consider_ld_for_removal && cr_pb) {
> > > +        consider_ld_for_removal = !ha_chassis_group_contains(
> > > +            cr_pb->ha_chassis_group, b_ctx_in->chassis_rec);
> > > +    }
> > > +
> > > +    if (!consider_ld_for_removal) {
> > > +        return true;
> > > +    }
> > > +
> > > +    int depth = 0;
> > > +
> > > +    bool is_relevant = local_datapath_is_relevant(
> > > +        ld, NULL, b_ctx_out->local_datapaths,
> > > +        &depth, b_ctx_in->chassis_rec,
> > > +        b_ctx_in->sbrec_port_binding_by_name);
> > > +
> > > +    if (depth >= 100) {
> > > +        /* datapaths are too deeply nested.  Fall back to recompute. */
> > > +        return false;
> > > +    }
> > > +
> > > +    if (!is_relevant) {
> > > +        /* This 'ld' can be removed from the local datapaths as
> > > +            *   - its a router datapath and
> > > +            *   - it has no peers locally. */
> > > +        local_datapath_remove_and_destroy(ld, b_ctx_out->local_datapaths,
> > > +                                          b_ctx_out->tracked_dp_bindings);
> > > +        *cleanedup = true;
> > > +    }
> > > +
> > > +    return true;
> > > +}
> > > +
> > > +static bool
> > > +local_datapath_is_relevant(struct local_datapath *ld,
> > > +                           struct local_datapath *ignore_peer_ld,
> > > +                           struct hmap *local_datapaths, int *depth,
> > > +                           const struct sbrec_chassis *chassis,
> > > +                           struct ovsdb_idl_index *sbrec_pb_by_name)
> > > +{
> > > +    if (!sset_is_empty(&ld->claimed_lports) ||
> > > +        !shash_is_empty(&ld->external_ports) ||
> > > +        !shash_is_empty(&ld->multichassis_ports) ||
> > > +        ld->vtep_port) {
> > > +        return true;
> > > +    }
> > > +
> > > +    bool relevant = false;
> > > +
> > > +    if (*depth >= 100) {
> > > +        static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> > > +        VLOG_WARN_RL(&rl, "datapaths nested too deep");
> > > +        return true;
> > > +    }
> > > +
> > > +    for (size_t i = 0; i < ld->n_peer_ports && !relevant; i++) {
> > > +        const struct sbrec_port_binding *remote = ld->peer_ports[i].remote;
> > > +        const struct sbrec_port_binding *local = ld->peer_ports[i].local;
> > > +
> > > +        if (is_patch_pb_chassis_relevant(local, chassis,
> > > +                                         sbrec_pb_by_name)) {
> > > +            return  true;
> > > +        }
> > > +
> > > +        if (is_patch_pb_chassis_relevant(remote, chassis,
> > > +                                         sbrec_pb_by_name)) {
> > > +            return  true;
> > > +        }
> > > +
> > > +        struct local_datapath *peer_ld;
> > > +        uint32_t remote_peer_ld_key;
> > > +        remote_peer_ld_key = ld->peer_ports[i].remote->datapath->tunnel_key;
> > > +        peer_ld = get_local_datapath(local_datapaths, remote_peer_ld_key);
> > > +        if (peer_ld && !peer_ld->has_only_dgp_peer_ports &&
> > > +            peer_ld != ignore_peer_ld) {
> > > +            *depth = *depth + 1;
> > > +            relevant = local_datapath_is_relevant(peer_ld, ld,
> > > +                                                  local_datapaths, depth,
> > > +                                                  chassis, sbrec_pb_by_name);
> > > +        }
> > > +    }
> > > +
> > > +    return relevant;
> > > +}
> > > diff --git a/controller/binding.h b/controller/binding.h
> > > index d13ae36c79..a4346c3e10 100644
> > > --- a/controller/binding.h
> > > +++ b/controller/binding.h
> > > @@ -201,6 +201,8 @@ bool binding_handle_port_binding_changes(struct binding_ctx_in *,
> > >  void binding_tracked_dp_destroy(struct hmap *tracked_datapaths);
> > >
> > >  void binding_dump_local_bindings(struct local_binding_data *, struct ds *);
> > > +void binding_dump_local_datapaths(struct hmap *local_datapaths,
> > > +                                  struct ds *out_data);
> > >
> > >  void binding_dump_related_lports(struct related_lports *related_lports,
> > >                                   struct ds *);
> > > diff --git a/controller/local_data.c b/controller/local_data.c
> > > index e19b2bf865..bb5740f1d6 100644
> > > --- a/controller/local_data.c
> > > +++ b/controller/local_data.c
> > > @@ -53,6 +53,13 @@ static struct tracked_datapath *tracked_datapath_create(
> > >
> > >  static bool datapath_is_switch(const struct sbrec_datapath_binding *);
> > >  static bool datapath_is_transit_switch(const struct sbrec_datapath_binding *);
> > > +static bool datapath_has_only_dgp_peer_ports(
> > > +    const struct sbrec_datapath_binding *);
> > > +static void local_datapath_remove_and_destroy__(
> > > +    struct local_datapath *ld,
> > > +    const struct sbrec_port_binding *ignore_peer_port,
> > > +    struct hmap *local_datapaths,
> > > +    struct hmap *tracked_datapaths);
> > >
> > >  static uint64_t local_datapath_usage;
> > >
> > > @@ -86,6 +93,7 @@ local_datapath_alloc(const struct sbrec_datapath_binding *dp)
> > >      ld->datapath = dp;
> > >      ld->is_switch = datapath_is_switch(dp);
> > >      ld->is_transit_switch = datapath_is_transit_switch(dp);
> > > +    ld->has_only_dgp_peer_ports = datapath_has_only_dgp_peer_ports(dp);
> > >      shash_init(&ld->external_ports);
> > >      shash_init(&ld->multichassis_ports);
> > >      sset_init(&ld->claimed_lports);
> > > @@ -132,6 +140,14 @@ local_datapath_destroy(struct local_datapath *ld)
> > >      free(ld);
> > >  }
> > >
> > > +void local_datapath_remove_and_destroy(struct local_datapath *ld,
> > > +                                       struct hmap *local_datapaths,
> > > +                                       struct hmap *tracked_datapaths)
> > > +{
> > > +    local_datapath_remove_and_destroy__(ld, NULL, local_datapaths,
> > > +                                        tracked_datapaths);
> > > +}
> > > +
> > >  /* Checks if pb is running on local gw router or pb is a patch port
> > >   * and the peer datapath should be added to local datapaths. */
> > >  bool
> > > @@ -226,12 +242,12 @@ add_local_datapath_peer_port(
> > >          get_local_datapath(local_datapaths,
> > >                             peer->datapath->tunnel_key);
> > >      if (!peer_ld) {
> > > -        add_local_datapath__(sbrec_datapath_binding_by_key,
> > > -                             sbrec_port_binding_by_datapath,
> > > -                             sbrec_port_binding_by_name, 1,
> > > -                             peer->datapath, chassis, local_datapaths,
> > > -                             tracked_datapaths);
> > > -        return;
> > > +        peer_ld = add_local_datapath__(sbrec_datapath_binding_by_key,
> > > +                                       sbrec_port_binding_by_datapath,
> > > +                                       sbrec_port_binding_by_name, 1,
> > > +                                       peer->datapath, chassis,
> > > +                                       local_datapaths,
> > > +                                       tracked_datapaths);
> > >      }
> > >
> > >      local_datapath_peer_port_add(peer_ld, peer, pb);
> > > @@ -613,6 +629,17 @@ add_local_datapath__(struct ovsdb_idl_index *sbrec_datapath_binding_by_key,
> > >                               tracked_datapaths);
> > >      }
> > >
> > > +    if (ld->has_only_dgp_peer_ports) {
> > > +        /* If this flag is set, it means this 'switch' datapath has
> > > +         *  - one ore many localnet ports.
> > > +         *  - all the router ports it is connected to are
> > > +         *    distributed gateway ports (DGPs).
> > > +         * There is no need to add the routers of the dgps to
> > > +         * the local datapaths.
> > > +         * */
> > > +        return ld;
> > > +    }
> > > +
> > >      if (depth >= 100) {
> > >          static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 1);
> > >          VLOG_WARN_RL(&rl, "datapaths nested too deep");
> > > @@ -705,6 +732,13 @@ datapath_is_transit_switch(const struct sbrec_datapath_binding *ldp)
> > >      return smap_get(&ldp->external_ids, "interconn-ts") != NULL;
> > >  }
> > >
> > > +static bool
> > > +datapath_has_only_dgp_peer_ports(const struct sbrec_datapath_binding *ldp)
> > > +{
> > > +    return datapath_is_switch(ldp) &&
> > > +           smap_get_bool(&ldp->external_ids, "only_dgp_peer_ports", false);
> > > +}
> > > +
> > >  bool
> > >  lb_is_local(const struct sbrec_load_balancer *sbrec_lb,
> > >              const struct hmap *local_datapaths)
> > > @@ -745,3 +779,41 @@ lb_is_local(const struct sbrec_load_balancer *sbrec_lb,
> > >
> > >      return false;
> > >  }
> > > +
> > > +static void
> > > +local_datapath_remove_and_destroy__(struct local_datapath *ld,
> > > +                                    const struct sbrec_port_binding *ignore_pb,
> > > +                                    struct hmap *local_datapaths,
> > > +                                    struct hmap *tracked_datapaths)
> > > +{
> > > +    for (size_t i = 0; i < ld->n_peer_ports; i++) {
> > > +        const struct sbrec_port_binding *remote = ld->peer_ports[i].remote;
> > > +        const struct sbrec_port_binding *local = ld->peer_ports[i].local;
> > > +
> > > +        if (local == ignore_pb) {
> > > +            continue;
> > > +        }
> > > +
> > > +        struct local_datapath *peer_ld;
> > > +        uint32_t remote_peer_ld_key;
> > > +
> > > +        remote_peer_ld_key = ld->peer_ports[i].remote->datapath->tunnel_key;
> > > +        peer_ld = get_local_datapath(local_datapaths, remote_peer_ld_key);
> > > +        if (peer_ld && !peer_ld->has_only_dgp_peer_ports) {
> > > +            local_datapath_remove_and_destroy__(peer_ld, remote,
> > > +                                                local_datapaths,
> > > +                                                tracked_datapaths);
> > > +        } else if (peer_ld && peer_ld->has_only_dgp_peer_ports) {
> > > +            remove_local_datapath_peer_port(ld->peer_ports[i].remote,
> > > +                                            peer_ld, local_datapaths);
> > > +        }
> > > +    }
> > > +
> > > +    hmap_remove(local_datapaths, &ld->hmap_node);
> > > +    if (tracked_datapaths) {
> > > +        tracked_datapath_add(ld->datapath, TRACKED_RESOURCE_REMOVED,
> > > +                             tracked_datapaths);
> > > +    }
> > > +
> > > +    local_datapath_destroy(ld);
> > > +}
> > > diff --git a/controller/local_data.h b/controller/local_data.h
> > > index 857d63a51d..ef01f4259e 100644
> > > --- a/controller/local_data.h
> > > +++ b/controller/local_data.h
> > > @@ -46,6 +46,8 @@ struct local_datapath {
> > >      const struct sbrec_datapath_binding *datapath;
> > >      bool is_switch;
> > >      bool is_transit_switch;
> > > +    /* Valid only for 'is_switch' local datapath. */
> > > +    bool has_only_dgp_peer_ports;
> > >
> > >      /* The localnet port in this datapath, if any (at most one is allowed). */
> > >      const struct sbrec_port_binding *localnet_port;
> > > @@ -91,6 +93,10 @@ struct local_datapath * add_local_datapath(
> > >
> > >  void local_datapaths_destroy(struct hmap *local_datapaths);
> > >  void local_datapath_destroy(struct local_datapath *ld);
> > > +void local_datapath_remove_and_destroy(struct local_datapath *,
> > > +                                       struct hmap *local_datapaths,
> > > +                                       struct hmap *tracked_datapaths);
> > > +
> > >  void add_local_datapath_peer_port(
> > >      const struct sbrec_port_binding *,
> > >      const struct sbrec_chassis *,
> > > diff --git a/controller/lport.c b/controller/lport.c
> > > index 8bc230e896..f164803528 100644
> > > --- a/controller/lport.c
> > > +++ b/controller/lport.c
> > > @@ -102,6 +102,18 @@ lport_get_l3gw_peer(const struct sbrec_port_binding *pb,
> > >      return get_peer_lport(pb, sbrec_port_binding_by_name);
> > >  }
> > >
> > > +const struct sbrec_port_binding *
> > > +lport_get_cr_port(struct ovsdb_idl_index *sbrec_port_binding_by_name,
> > > +                  const struct sbrec_port_binding *pb)
> > > +{
> > > +    const char *crp = smap_get(&pb->options, "chassis-redirect-port");
> > > +    if (crp) {
> > > +        return lport_lookup_by_name(sbrec_port_binding_by_name, crp);
> > > +    }
> > > +
> > > +    return NULL;
> > > +}
> > > +
> > >  enum can_bind
> > >  lport_can_bind_on_this_chassis(const struct sbrec_chassis *chassis_rec,
> > >                                 const struct sbrec_port_binding *pb)
> > > diff --git a/controller/lport.h b/controller/lport.h
> > > index 3a58ebe39e..069bd59096 100644
> > > --- a/controller/lport.h
> > > +++ b/controller/lport.h
> > > @@ -73,4 +73,8 @@ const struct sbrec_port_binding *lport_get_l3gw_peer(
> > >  bool
> > >  lport_is_activated_by_activation_strategy(const struct sbrec_port_binding *pb,
> > >                                            const struct sbrec_chassis *chassis);
> > > +const struct sbrec_port_binding *lport_get_cr_port(
> > > +    struct ovsdb_idl_index *sbrec_port_binding_by_name,
> > > +    const struct sbrec_port_binding *);
> > > +
> > >  #endif /* controller/lport.h */
> > > diff --git a/controller/ovn-controller.c b/controller/ovn-controller.c
> > > index da942abaa9..dc8ceec076 100644
> > > --- a/controller/ovn-controller.c
> > > +++ b/controller/ovn-controller.c
> > > @@ -101,6 +101,7 @@ static unixctl_cb_func debug_pause_execution;
> > >  static unixctl_cb_func debug_resume_execution;
> > >  static unixctl_cb_func debug_status_execution;
> > >  static unixctl_cb_func debug_dump_local_bindings;
> > > +static unixctl_cb_func debug_dump_local_datapaths;
> > >  static unixctl_cb_func debug_dump_related_lports;
> > >  static unixctl_cb_func debug_dump_local_template_vars;
> > >  static unixctl_cb_func debug_dump_local_mac_bindings;
> > > @@ -1688,6 +1689,22 @@ runtime_data_sb_datapath_binding_handler(struct engine_node *node OVS_UNUSED,
> > >                  return false;
> > >              }
> > >          }
> > > +
> > > +        if (sbrec_datapath_binding_is_updated(
> > > +                dp, SBREC_DATAPATH_BINDING_COL_EXTERNAL_IDS) &&
> > > +            !sbrec_datapath_binding_is_new(dp)) {
> > > +            struct local_datapath *ld =
> > > +                get_local_datapath(&rt_data->local_datapaths,
> > > +                                   dp->tunnel_key);
> > > +                if (ld && ld->is_switch) {
> > > +                    bool only_dgp_peer_ports =
> > > +                        smap_get_bool(&dp->external_ids, "only_dgp_peer_ports",
> > > +                                      false);
> > > +                    if (ld->has_only_dgp_peer_ports != only_dgp_peer_ports) {
> > > +                        return false;
> > > +                    }
> > > +                }
> > > +        }
> > >      }
> > >
> > >      return true;
> > > @@ -4357,6 +4374,12 @@ lflow_output_runtime_data_handler(struct engine_node *node,
> > >      init_lflow_ctx(node, fo, &l_ctx_in, &l_ctx_out);
> > >
> > >      struct tracked_datapath *tdp;
> > > +    HMAP_FOR_EACH (tdp, node, tracked_dp_bindings) {
> > > +        if (tdp->tracked_type == TRACKED_RESOURCE_REMOVED) {
> > > +            return false;
> > > +        }
> > > +    }
> > > +
> > >      HMAP_FOR_EACH (tdp, node, tracked_dp_bindings) {
> > >          if (tdp->tracked_type == TRACKED_RESOURCE_NEW) {
> > >              if (!lflow_add_flows_for_datapath(tdp->dp, &l_ctx_in,
> > > @@ -5552,6 +5575,10 @@ main(int argc, char *argv[])
> > >                               debug_dump_local_bindings,
> > >                               &runtime_data->lbinding_data);
> > >
> > > +    unixctl_command_register("debug/dump-local-datapaths", "", 0, 0,
> > > +                             debug_dump_local_datapaths,
> > > +                             &runtime_data->local_datapaths);
> > > +
> > >      unixctl_command_register("debug/dump-related-ports", "", 0, 0,
> > >                               debug_dump_related_lports,
> > >                               &runtime_data->related_lports);
> > > @@ -6517,6 +6544,17 @@ debug_dump_local_bindings(struct unixctl_conn *conn, int argc OVS_UNUSED,
> > >      ds_destroy(&binding_data);
> > >  }
> > >
> > > +static void
> > > +debug_dump_local_datapaths(struct unixctl_conn *conn, int argc OVS_UNUSED,
> > > +                           const char *argv[] OVS_UNUSED,
> > > +                           void *local_datapaths)
> > > +{
> > > +    struct ds local_dps_data = DS_EMPTY_INITIALIZER;
> > > +    binding_dump_local_datapaths(local_datapaths, &local_dps_data);
> > > +    unixctl_command_reply(conn, ds_cstr(&local_dps_data));
> > > +    ds_destroy(&local_dps_data);
> > > +}
> > > +
> > >  static void
> > >  debug_dump_related_lports(struct unixctl_conn *conn, int argc OVS_UNUSED,
> > >                            const char *argv[] OVS_UNUSED, void *related_lports)
> > > diff --git a/tests/multinode.at b/tests/multinode.at
> > > index c1bd3123ac..b828f16135 100644
> > > --- a/tests/multinode.at
> > > +++ b/tests/multinode.at
> > > @@ -2776,3 +2776,181 @@ for i in 1 2; do
> > >  done
> > >
> > >  AT_CLEANUP
> > > +
> > > +AT_SETUP([ovn multinode - only_dgp_peer_ports provider switch functionality])
> > > +
> > > +# Check that ovn-fake-multinode setup is up and running
> > > +check_fake_multinode_setup
> > > +
> > > +# Delete the multinode NB and OVS resources before starting the test.
> > > +cleanup_multinode_resources
> > > +
> > > +check multinode_nbctl ls-add sw0
> > > +check multinode_nbctl lsp-add sw0 sw0-port1
> > > +check multinode_nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:03 10.0.0.3 1000::3"
> > > +check multinode_nbctl lsp-add sw0 sw0-port2
> > > +check multinode_nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:04 10.0.0.4 1000::4"
> > > +
> > > +check multinode_nbctl lr-add lr0
> > > +check multinode_nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 1000::1/64
> > > +check multinode_nbctl lsp-add sw0 sw0-lr0
> > > +check multinode_nbctl lsp-set-type sw0-lr0 router
> > > +check multinode_nbctl lsp-set-addresses sw0-lr0 router
> > > +check multinode_nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> > > +
> > > +check multinode_nbctl ls-add public
> > > +check multinode_nbctl lrp-add lr0 lr0-public 00:00:20:20:12:13 172.16.1.100/24 2000::1/64
> > > +check multinode_nbctl lsp-add public public-lr0
> > > +check multinode_nbctl lsp-set-type public-lr0 router
> > > +check multinode_nbctl lsp-set-addresses public-lr0 router
> > > +check multinode_nbctl lsp-set-options public-lr0 router-port=lr0-public
> > > +
> > > +# localnet port
> > > +check multinode_nbctl lsp-add public ln-public
> > > +check multinode_nbctl lsp-set-type ln-public localnet
> > > +check multinode_nbctl lsp-set-addresses ln-public unknown
> > > +check multinode_nbctl lsp-set-options ln-public network_name=public
> > > +
> > > +check multinode_nbctl lrp-set-gateway-chassis lr0-public ovn-gw-1 20
> > > +check multinode_nbctl lr-nat-add lr0 snat 172.16.1.100 10.0.0.0/24
> > > +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 172.16.1.110 10.0.0.3 sw0-port1 50:54:00:00:00:03
> > > +check multinode_nbctl lr-nat-add lr0 snat 2000::1 1000::/64
> > > +check multinode_nbctl lr-nat-add lr0 dnat_and_snat 2000::2 1000::3 sw0-port1 50:54:00:00:00:03
> > > +
> > > +check multinode_nbctl --wait=hv sync
> > > +
> > > +check multinode_nbctl ls-add sw1
> > > +check multinode_nbctl lsp-add sw1 sw1-port1
> > > +check multinode_nbctl lsp-set-addresses sw1-port1 "40:54:00:00:00:03 20.0.0.3 2000::3"
> > > +
> > > +check multinode_nbctl lr-add lr1
> > > +check multinode_nbctl lrp-add lr1 lr1-sw1 00:00:01:00:ef:01 20.0.0.1/24 2000::a/64
> > > +check multinode_nbctl lsp-add sw1 sw1-lr1
> > > +check multinode_nbctl lsp-set-type sw1-lr1 router
> > > +check multinode_nbctl lsp-set-addresses sw1-lr1 router
> > > +check multinode_nbctl lsp-set-options sw1-lr1 router-port=lr1-sw1
> > > +
> > > +check multinode_nbctl lrp-add lr1 lr1-public 00:00:20:30:22:13 172.16.1.101/24
> > > +check multinode_nbctl lsp-add public public-lr1
> > > +check multinode_nbctl lsp-set-type public-lr1 router
> > > +check multinode_nbctl lsp-set-addresses public-lr1 router
> > > +check multinode_nbctl lsp-set-options public-lr1 router-port=lr1-public
> > > +
> > > +check multinode_nbctl lr-nat-add lr1 snat 172.16.1.101 20.0.0.0/24
> > > +check multinode_nbctl lr-nat-add lr1 dnat_and_snat 172.16.1.120 20.0.0.3
> > > +check multinode_nbctl lrp-set-gateway-chassis lr1-public ovn-gw-1 20
> > > +
> > > +check multinode_nbctl --wait=hv sync
> > > +
> > > +m_as ovn-chassis-1 /data/create_fake_vm.sh sw0-port1 sw0p1 50:54:00:00:00:03 1342 10.0.0.3 24 10.0.0.1 1000::3/64 1000::a
> > > +m_as ovn-chassis-2 /data/create_fake_vm.sh sw1-port1 sw1p1 40:54:00:00:00:03 1342 20.0.0.3 24 20.0.0.1 2000::4/64 1000::a
> > > +
> > > +m_wait_for_ports_up sw0-port1
> > > +m_wait_for_ports_up sw1-port1
> > > +
> > > +m_as ovn-central-az1-1 ovn-sbctl show
> > > +
> > > +m_as ovn-chassis-1 ovn-appctl  debug/dump-local-datapaths | sort
> > > +m_as ovn-chassis-2 ovn-appctl  debug/dump-local-datapaths | sort
> > > +
> > > +AT_CHECK([m_as ovn-chassis-1 ovn-appctl  debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([m_as ovn-chassis-2 ovn-appctl  debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr1, type: router
> > > +Datapath: sw1, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([m_as ovn-gw-1 ovn-appctl  debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: lr1, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Datapath: sw1, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +# ping lr0-public IP - 172.168.0.100
> > > +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.16.1.100 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +# ping lr1-public IP - 172.168.0.101 from sw0p1
> > > +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.16.1.101 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +# ping public ip of sw1-port1 - 172.16.1.120 from sw0p1
> > > +M_NS_CHECK_EXEC([ovn-chassis-1], [sw0p1], [ping -q -c 3 -i 0.3 -w 2 172.16.1.120 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +# ping public ip of sw0-port1 - 172.16.1.110 from sw1p1
> > > +M_NS_CHECK_EXEC([ovn-chassis-2], [sw1p1], [ping -q -c 3 -i 0.3 -w 2 172.16.1.110 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +# Bind sw0-port2 on chassis-2
> > > +m_as ovn-chassis-2 /data/create_fake_vm.sh sw0-port2 sw0p2 50:54:00:00:00:04 1342 10.0.0.4 24 10.0.0.1 1000::4/64 1000::a
> > > +m_wait_for_ports_up sw0-port2
> > > +
> > > +AT_CHECK([m_as ovn-chassis-2 ovn-appctl  debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: lr1, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Datapath: sw1, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +# ping public ip of sw0-port1 - 172.16.1.110 from sw0p2
> > > +M_NS_CHECK_EXEC([ovn-chassis-2], [sw0p2], [ping -q -c 3 -i 0.3 -w 2 172.16.1.110 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +# ping public ip of sw1-port1 - 172.16.1.120 from sw0p2
> > > +M_NS_CHECK_EXEC([ovn-chassis-2], [sw0p2], [ping -q -c 3 -i 0.3 -w 2 172.16.1.120 | FORMAT_PING], \
> > > +[0], [dnl
> > > +3 packets transmitted, 3 received, 0% packet loss, time 0ms
> > > +])
> > > +
> > > +# Create a normal router port in public with its peer as a normal distributed router port.
> > > +check multinode_nbctl lsp-add public public-lr2
> > > +check multinode_nbctl lsp-set-type public-lr2 router
> > > +check multinode_nbctl lsp-set-addresses public-lr2 router
> > > +check multinode_nbctl lsp-set-options public-lr2 router-port=lr2-public
> > > +check multinode_nbctl lr-add lr2
> > > +check multinode_nbctl lrp-add lr2 lr2-public 00:00:41:00:1f:61 172.16.1.102/24 3000::a/64
> > > +
> > > +check multinode_nbctl --wait=hv sync
> > > +AT_CHECK([m_as ovn-chassis-1 ovn-appctl  debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: lr1, type: router
> > > +Datapath: lr2, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Datapath: sw1, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([m_as ovn-chassis-2 ovn-appctl  debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: lr1, type: router
> > > +Datapath: lr2, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Datapath: sw1, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CLEANUP
> > > diff --git a/tests/ovn-performance.at b/tests/ovn-performance.at
> > > index 7d480c20c8..a003fc36bc 100644
> > > --- a/tests/ovn-performance.at
> > > +++ b/tests/ovn-performance.at
> > > @@ -479,7 +479,7 @@ OVN_CONTROLLER_EXPECT_NO_HIT(
> > >  )
> > >
> > >  OVN_CONTROLLER_EXPECT_HIT_COND(
> > > -    [hv1 hv2 hv3 hv4 hv5], [lflow_run], [=0 =0 >0 =0 =0],
> > > +    [hv1 hv2 hv3 hv4 hv5], [lflow_run], [>0 >0 >0 =0 =0],
> > >      [ovn-nbctl --wait=hv lrp-set-gateway-chassis lr1-public hv3 30 && ovn-nbctl --wait=hv sync]
> > >  )
> > >
> > > @@ -552,8 +552,8 @@ hv5_ch=$(ovn-sbctl --bare --columns _uuid list chassis hv5)
> > >  OVS_WAIT_UNTIL([ovn-sbctl find port_binding logical_port=cr-lr1-public chassis=$hv5_ch])
> > >  check ovn-nbctl --wait=hv sync
> > >  # Delete hv5 from gateway chassis. There should be no lflow_run.
> > > -OVN_CONTROLLER_EXPECT_NO_HIT(
> > > -    [hv1 hv2 hv3 hv4 hv5], [lflow_run],
> > > +OVN_CONTROLLER_EXPECT_HIT_COND(
> > > +    [hv1 hv2 hv3 hv4 hv5], [lflow_run], [=0 =0 =0 =0 =0]
> > >      [ovn-nbctl --wait=hv lrp-del-gateway-chassis lr1-public hv5]
> > >  )
> > >
> > > diff --git a/tests/ovn.at b/tests/ovn.at
> > > index d105ed2535..d4e87212a5 100644
> > > --- a/tests/ovn.at
> > > +++ b/tests/ovn.at
> > > @@ -41481,6 +41481,859 @@ OVN_CLEANUP([hv1])
> > >  AT_CLEANUP
> > >  ])
> > >
> > > +OVN_FOR_EACH_NORTHD_NO_HV([
> > > +AT_SETUP([ovn-controller -- only_dgp_peer_ports flag in SB datapath_binding])
> > > +AT_KEYWORDS([multiple-l3dgw-ports])
> > > +ovn_start
> > > +net_add n1
> > > +sim_add hv1
> > > +as hv1
> > > +check ovs-vsctl add-br br-phys
> > > +ovn_attach n1 br-phys 192.168.0.1
> > > +
> > > +sim_add hv2
> > > +as hv2
> > > +check ovs-vsctl add-br br-phys
> > > +ovn_attach n1 br-phys 192.168.0.2
> > > +
> > > +sim_add gw1
> > > +as gw1
> > > +check ovs-vsctl add-br br-phys
> > > +ovn_attach n1 br-phys 192.168.0.3
> > > +
> > > +sim_add gw2
> > > +as gw2
> > > +check ovs-vsctl add-br br-phys
> > > +ovn_attach n1 br-phys 192.168.0.4
> > > +
> > > +check ovn-nbctl ls-add sw0
> > > +check ovn-nbctl lsp-add sw0 sw0-port1
> > > +check ovn-nbctl lsp-set-addresses sw0-port1 "50:54:00:00:00:01 10.0.0.3 1000::3"
> > > +check ovn-nbctl lsp-add sw0 sw0-port2
> > > +check ovn-nbctl lsp-set-addresses sw0-port2 "50:54:00:00:00:02 10.0.0.4 1000::4"
> > > +
> > > +check ovn-nbctl lr-add lr0
> > > +check ovn-nbctl lrp-add lr0 lr0-sw0 00:00:00:00:ff:01 10.0.0.1/24 1000::1/64
> > > +check ovn-nbctl lsp-add sw0 sw0-lr0
> > > +check ovn-nbctl lsp-set-type sw0-lr0 router
> > > +check ovn-nbctl lsp-set-addresses sw0-lr0 router
> > > +check ovn-nbctl lsp-set-options sw0-lr0 router-port=lr0-sw0
> > > +
> > > +check ovn-nbctl ls-add public
> > > +check ovn-nbctl lrp-add lr0 lr0-public 00:00:20:20:12:13 172.168.0.100/24 2000::1/64
> > > +check ovn-nbctl lsp-add public public-lr0
> > > +check ovn-nbctl lsp-set-type public-lr0 router
> > > +check ovn-nbctl lsp-set-addresses public-lr0 router
> > > +check ovn-nbctl lsp-set-options public-lr0 router-port=lr0-public
> > > +
> > > +# localnet port
> > > +check ovn-nbctl lsp-add public ln-public
> > > +check ovn-nbctl lsp-set-type ln-public localnet
> > > +check ovn-nbctl lsp-set-addresses ln-public unknown
> > > +check ovn-nbctl lsp-set-options ln-public network_name=phys
> > > +
> > > +check ovn-nbctl lrp-set-gateway-chassis lr0-public gw1 20
> > > +check ovn-nbctl lr-nat-add lr0 snat 172.168.0.100 10.0.0.0/24
> > > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 172.168.0.110 10.0.0.4 sw0-port2 f0:00:00:01:02:04
> > > +check ovn-nbctl lr-nat-add lr0 snat 2000::1 1000::/64
> > > +check ovn-nbctl lr-nat-add lr0 dnat_and_snat 2000::2 1000::4 sw0-port2 f0:00:00:01:02:04
> > > +
> > > +check ovn-nbctl --wait=hv sync
> > > +
> > > +sw0_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=sw0))
> > > +lr0_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=lr0))
> > > +public_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=public))
> > > +
> > > +check_offlows_for_datapath() {
> > > +    hv=$1
> > > +    dp_key=$2
> > > +    should_be_present=$3
> > > +
> > > +    if [[ "$should_be_present" == "yes" ]]; then
> > > +        echo "Flows should be present for hv - $hv : datapath - $dp_key"
> > > +        OVS_WAIT_UNTIL(
> > > +            [test $(as $hv ovs-ofctl dump-flows br-int | grep -c metadata=0x$dp_key) -gt 0]
> > > +        )
> > > +    else
> > > +        echo "Flows should NOT be present for hv - $hv : datapath - $dp_key"
> > > +        OVS_WAIT_UNTIL(
> > > +            [test $(as $hv ovs-ofctl dump-flows br-int | grep -c metadata=0x$dp_key) -eq 0]
> > > +        )
> > > +    fi
> > > +}
> > > +
> > > +AT_CHECK([ovn-sbctl get datapath_binding public external_ids:only_dgp_peer_ports], [0], [dnl
> > > +"true"
> > > +])
> > > +
> > > +check_offlows_for_datapath hv1 $sw0_dp_key no
> > > +check_offlows_for_datapath hv1 $lr0_dp_key no
> > > +check_offlows_for_datapath hv1 $public_dp_key no
> > > +
> > > +check_offlows_for_datapath hv2 $sw0_dp_key no
> > > +check_offlows_for_datapath hv2 $lr0_dp_key no
> > > +check_offlows_for_datapath hv2 $public_dp_key no
> > > +
> > > +check_offlows_for_datapath gw1 $sw0_dp_key yes
> > > +check_offlows_for_datapath gw1 $lr0_dp_key yes
> > > +check_offlows_for_datapath gw1 $public_dp_key yes
> > > +
> > > +check_offlows_for_datapath gw2 $sw0_dp_key no
> > > +check_offlows_for_datapath gw2 $lr0_dp_key no
> > > +check_offlows_for_datapath gw2 $public_dp_key no
> > > +
> > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> > > +Local datapaths:
> > > +])
> > > +
> > > +# Create a VIF on hv1 for sw0-port1
> > > +AS_BOX([create a VIF on hv1 for sw0-port1])
> > > +
> > > +as hv1
> > > +ovs-vsctl -- add-port br-int hv1-vif1 -- \
> > > +    set interface hv1-vif1 external-ids:iface-id=sw0-port1 \
> > > +    options:tx_pcap=hv1/vif1-tx.pcap \
> > > +    options:rxq_pcap=hv1/vif1-rx.pcap \
> > > +    ofport-request=1
> > > +
> > > +wait_for_ports_up sw0-port1
> > > +
> > > +AS_BOX([Create a VIF on hv1 for sw0-port1 - hv1 should have flows for sw0, lr0 and public])
> > > +
> > > +check_offlows_for_datapath hv1 $sw0_dp_key yes
> > > +check_offlows_for_datapath hv1 $lr0_dp_key yes
> > > +check_offlows_for_datapath hv1 $public_dp_key yes
> > > +
> > > +AS_BOX([hv2 should NOT have flows for sw0, lr0 and public])
> > > +check_offlows_for_datapath hv2 $sw0_dp_key no
> > > +check_offlows_for_datapath hv2 $lr0_dp_key no
> > > +check_offlows_for_datapath hv2 $public_dp_key no
> > > +
> > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> > > +Local datapaths:
> > > +])
> > > +
> > > +AS_BOX([create a switch sw1 and router lr1, attach both and attach lr1 to public])
> > > +
> > > +check ovn-nbctl ls-add sw1
> > > +check ovn-nbctl lsp-add sw1 sw1-port1
> > > +check ovn-nbctl lsp-set-addresses sw1-port1 "60:54:00:00:00:01 20.0.0.3"
> > > +
> > > +check ovn-nbctl lr-add lr1
> > > +check ovn-nbctl lrp-add lr1 lr1-sw1 00:00:01:00:ef:01 20.0.0.1/24
> > > +check ovn-nbctl lsp-add sw1 sw1-lr1
> > > +check ovn-nbctl lsp-set-type sw1-lr1 router
> > > +check ovn-nbctl lsp-set-addresses sw1-lr1 router
> > > +check ovn-nbctl lsp-set-options sw1-lr1 router-port=lr1-sw1
> > > +
> > > +check ovn-nbctl lrp-add lr1 lr1-public 00:00:20:30:22:13 172.168.0.101/24
> > > +check ovn-nbctl lsp-add public public-lr1
> > > +check ovn-nbctl lsp-set-type public-lr1 router
> > > +check ovn-nbctl lsp-set-addresses public-lr1 router
> > > +check ovn-nbctl lsp-set-options public-lr1 router-port=lr1-public
> > > +
> > > +sw1_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=sw1))
> > > +lr1_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=lr1))
> > > +
> > > +check ovn-nbctl lr-nat-add lr1 snat 172.168.0.101 20.0.0.0/24
> > > +check ovn-nbctl lr-nat-add lr1 dnat_and_snat 172.168.0.140 20.0.0.3
> > > +
> > > +AS_BOX([create a switch sw2 and router lr2, attach both and attach lr2 to public])
> > > +
> > > +check ovn-nbctl ls-add sw2
> > > +check ovn-nbctl lsp-add sw2 sw2-port1
> > > +check ovn-nbctl lsp-set-addresses sw2-port1 "70:54:00:00:00:01 30.0.0.3"
> > > +
> > > +check ovn-nbctl lr-add lr2
> > > +check ovn-nbctl lrp-add lr2 lr2-sw2 00:00:02:00:ef:01 30.0.0.1/24
> > > +check ovn-nbctl lsp-add sw2 sw2-lr2
> > > +check ovn-nbctl lsp-set-type sw2-lr2 router
> > > +check ovn-nbctl lsp-set-addresses sw2-lr2 router
> > > +check ovn-nbctl lsp-set-options sw2-lr2 router-port=lr2-sw2
> > > +
> > > +check ovn-nbctl lrp-add lr2 lr2-public 00:00:20:40:22:53 172.168.0.102/24
> > > +check ovn-nbctl lsp-add public public-lr2
> > > +check ovn-nbctl lsp-set-type public-lr2 router
> > > +check ovn-nbctl lsp-set-addresses public-lr2 router
> > > +check ovn-nbctl lsp-set-options public-lr2 router-port=lr2-public
> > > +
> > > +sw2_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=sw2))
> > > +lr2_dp_key=$(printf "%x" $(fetch_column Datapath_Binding tunnel_key external_ids:name=lr2))
> > > +
> > > +check ovn-nbctl lr-nat-add lr2 snat 172.168.0.102 30.0.0.0/24
> > > +check ovn-nbctl lr-nat-add lr2 dnat_and_snat 172.168.0.150 30.0.0.3
> > > +
> > > +check ovn-nbctl --wait=hv sync
> > > +
> > > +# Since lr1-public is not a DGP,  public is not a "only_dgp_peer_ports".
> > > +AT_CHECK([ovn-sbctl get datapath_binding public external_ids:only_dgp_peer_ports], [1], [ignore], [ignore])
> > > +
> > > +check_offlows_for_datapath hv1 $sw1_dp_key yes
> > > +check_offlows_for_datapath hv1 $lr1_dp_key yes
> > > +check_offlows_for_datapath hv1 $sw2_dp_key yes
> > > +check_offlows_for_datapath hv1 $lr2_dp_key yes
> > > +
> > > +check_offlows_for_datapath hv2 $sw1_dp_key no
> > > +check_offlows_for_datapath hv2 $lr1_dp_key no
> > > +check_offlows_for_datapath hv2 $sw2_dp_key no
> > > +check_offlows_for_datapath hv2 $lr2_dp_key no
> > > +
> > > +check_offlows_for_datapath gw1 $sw1_dp_key yes
> > > +check_offlows_for_datapath gw1 $lr1_dp_key yes
> > > +check_offlows_for_datapath gw1 $sw2_dp_key yes
> > > +check_offlows_for_datapath gw1 $lr2_dp_key yes
> > > +
> > > +check_offlows_for_datapath gw2 $sw1_dp_key no
> > > +check_offlows_for_datapath gw2 $lr1_dp_key no
> > > +check_offlows_for_datapath gw2 $sw2_dp_key no
> > > +check_offlows_for_datapath gw2 $lr2_dp_key no
> > > +
> > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: lr1, type: router
> > > +Datapath: lr2, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Datapath: sw1, type: switch
> > > +Datapath: sw2, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: lr1, type: router
> > > +Datapath: lr2, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Datapath: sw1, type: switch
> > > +Datapath: sw2, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> > > +Local datapaths:
> > > +])
> > > +
> > > +AS_BOX([Set gw2 as gateway chassis for lr1-public and lr2-public])
> > > +check ovn-nbctl --wait=hv lrp-set-gateway-chassis lr1-public gw2 20
> > > +check ovn-nbctl --wait=hv lrp-set-gateway-chassis lr2-public gw2 30
> > > +wait_row_count Port_Binding 1 logical_port=cr-lr1-public
> > > +wait_row_count Port_Binding 1 logical_port=cr-lr2-public
> > > +
> > > +AT_CHECK([ovn-sbctl get datapath_binding public external_ids:only_dgp_peer_ports], [0], [dnl
> > > +"true"
> > > +])
> > > +
> > > +check ovn-nbctl --wait=hv sync
> > > +
> > > +check_offlows_for_datapath hv1 $sw0_dp_key yes
> > > +check_offlows_for_datapath hv1 $lr0_dp_key yes
> > > +check_offlows_for_datapath hv1 $public_dp_key yes
> > > +check_offlows_for_datapath hv1 $sw1_dp_key no
> > > +check_offlows_for_datapath hv1 $lr1_dp_key no
> > > +check_offlows_for_datapath hv1 $sw2_dp_key no
> > > +check_offlows_for_datapath hv1 $lr2_dp_key no
> > > +
> > > +check_offlows_for_datapath hv2 $sw0_dp_key no
> > > +check_offlows_for_datapath hv2 $lr0_dp_key no
> > > +check_offlows_for_datapath hv2 $public_dp_key no
> > > +check_offlows_for_datapath hv2 $sw1_dp_key no
> > > +check_offlows_for_datapath hv2 $lr1_dp_key no
> > > +check_offlows_for_datapath hv2 $sw2_dp_key no
> > > +check_offlows_for_datapath hv2 $lr2_dp_key no
> > > +
> > > +check_offlows_for_datapath gw1 $sw0_dp_key yes
> > > +check_offlows_for_datapath gw1 $lr0_dp_key yes
> > > +check_offlows_for_datapath gw1 $public_dp_key yes
> > > +check_offlows_for_datapath gw1 $sw1_dp_key no
> > > +check_offlows_for_datapath gw1 $lr1_dp_key no
> > > +check_offlows_for_datapath gw1 $sw2_dp_key no
> > > +check_offlows_for_datapath gw1 $lr2_dp_key no
> > > +
> > > +check_offlows_for_datapath gw2 $sw0_dp_key no
> > > +check_offlows_for_datapath gw2 $lr0_dp_key no
> > > +check_offlows_for_datapath hv1 $public_dp_key yes
> > > +check_offlows_for_datapath gw2 $sw1_dp_key yes
> > > +check_offlows_for_datapath gw2 $lr1_dp_key yes
> > > +check_offlows_for_datapath gw2 $sw2_dp_key yes
> > > +check_offlows_for_datapath gw2 $lr2_dp_key yes
> > > +
> > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths], [0], [dnl
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr1, type: router
> > > +Datapath: lr2, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw1, type: switch
> > > +Datapath: sw2, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AS_BOX([Create a VIF on hv2 for sw1-port1])
> > > +
> > > +as hv2
> > > +ovs-vsctl -- add-port br-int hv2-vif1 -- \
> > > +    set interface hv2-vif1 external-ids:iface-id=sw1-port1 \
> > > +    options:tx_pcap=hv2/vif1-tx.pcap \
> > > +    options:rxq_pcap=hv2/vif1-rx.pcap \
> > > +    ofport-request=1
> > > +
> > > +wait_for_ports_up sw1-port1
> > > +
> > > +check_offlows_for_datapath hv1 $sw0_dp_key yes
> > > +check_offlows_for_datapath hv1 $lr0_dp_key yes
> > > +check_offlows_for_datapath hv1 $public_dp_key yes
> > > +check_offlows_for_datapath hv1 $sw1_dp_key no
> > > +check_offlows_for_datapath hv1 $lr1_dp_key no
> > > +check_offlows_for_datapath hv1 $sw2_dp_key no
> > > +check_offlows_for_datapath hv1 $lr2_dp_key no
> > > +
> > > +check_offlows_for_datapath hv2 $sw0_dp_key no
> > > +check_offlows_for_datapath hv2 $lr0_dp_key no
> > > +
> > > +# Since there are no distributed dnat_and_snat entries
> > > +# in lr1, hv2 will not have "public" in its
> > > +# local datapaths.
> > > +check_offlows_for_datapath hv2 $public_dp_key no
> > > +check_offlows_for_datapath hv2 $sw1_dp_key yes
> > > +check_offlows_for_datapath hv2 $lr1_dp_key yes
> > > +check_offlows_for_datapath hv2 $sw2_dp_key no
> > > +check_offlows_for_datapath hv2 $lr2_dp_key no
> > > +
> > > +check_offlows_for_datapath gw1 $sw0_dp_key yes
> > > +check_offlows_for_datapath gw1 $lr0_dp_key yes
> > > +check_offlows_for_datapath gw1 $public_dp_key yes
> > > +check_offlows_for_datapath gw1 $sw1_dp_key no
> > > +check_offlows_for_datapath gw1 $lr1_dp_key no
> > > +check_offlows_for_datapath gw1 $sw2_dp_key no
> > > +check_offlows_for_datapath gw1 $lr2_dp_key no
> > > +
> > > +# gw2 should have sw1, lr1, sw2 and lr2 and public in its local datapaths.
> > > +check_offlows_for_datapath gw2 $sw0_dp_key no
> > > +check_offlows_for_datapath gw2 $lr0_dp_key no
> > > +check_offlows_for_datapath gw2 $public_dp_key yes
> > > +check_offlows_for_datapath gw2 $sw1_dp_key yes
> > > +check_offlows_for_datapath gw2 $lr1_dp_key yes
> > > +check_offlows_for_datapath gw2 $sw2_dp_key yes
> > > +check_offlows_for_datapath gw2 $lr2_dp_key yes
> > > +
> > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr1, type: router
> > > +Datapath: sw1, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr1, type: router
> > > +Datapath: lr2, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw1, type: switch
> > > +Datapath: sw2, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +# Add distributed dnat_and_snat in lr1.  hv2 should have
> > > +# public in its local datapaths.
> > > +AS_BOX([ Add distributed dnat_and_snat in lr1])
> > > +
> > > +check ovn-nbctl lr-nat-del lr1 dnat_and_snat
> > > +check ovn-nbctl --wait=hv lr-nat-add lr1 dnat_and_snat 172.168.0.140 20.0.0.3 sw1-port1 10:00:00:01:02:14
> > > +
> > > +check_offlows_for_datapath hv2 $sw0_dp_key no
> > > +check_offlows_for_datapath hv2 $lr0_dp_key no
> > > +check_offlows_for_datapath hv2 $public_dp_key yes
> > > +check_offlows_for_datapath hv2 $sw1_dp_key yes
> > > +check_offlows_for_datapath hv2 $lr1_dp_key yes
> > > +check_offlows_for_datapath hv2 $sw2_dp_key no
> > > +check_offlows_for_datapath hv2 $lr2_dp_key no
> > > +
> > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr1, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw1, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr1, type: router
> > > +Datapath: lr2, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw1, type: switch
> > > +Datapath: sw2, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AS_BOX([Create a VIF on hv2 for sw0-port2])
> > > +
> > > +as hv2
> > > +ovs-vsctl -- add-port br-int hv2-vif2 -- \
> > > +    set interface hv2-vif2 external-ids:iface-id=sw0-port2 \
> > > +    options:tx_pcap=hv2/vif2-tx.pcap \
> > > +    options:rxq_pcap=hv2/vif2-rx.pcap \
> > > +    ofport-request=2
> > > +
> > > +wait_for_ports_up sw0-port2
> > > +
> > > +check_offlows_for_datapath hv2 $sw0_dp_key yes
> > > +check_offlows_for_datapath hv2 $lr0_dp_key yes
> > > +check_offlows_for_datapath hv2 $public_dp_key yes
> > > +check_offlows_for_datapath hv2 $sw1_dp_key yes
> > > +check_offlows_for_datapath hv2 $lr1_dp_key yes
> > > +check_offlows_for_datapath hv2 $sw2_dp_key no
> > > +check_offlows_for_datapath hv2 $lr2_dp_key no
> > > +
> > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: lr1, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Datapath: sw1, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr1, type: router
> > > +Datapath: lr2, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw1, type: switch
> > > +Datapath: sw2, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AS_BOX([Delete the VIF for sw1-port1 in hv2])
> > > +
> > > +as hv2 ovs-vsctl del-port hv2-vif1
> > > +check ovn-nbctl --wait=hv sync
> > > +check_column "false" Port_Binding up logical_port=sw1-port1
> > > +
> > > +check_offlows_for_datapath hv2 $sw0_dp_key yes
> > > +check_offlows_for_datapath hv2 $lr0_dp_key yes
> > > +check_offlows_for_datapath hv2 $public_dp_key yes
> > > +check_offlows_for_datapath hv2 $sw1_dp_key no
> > > +check_offlows_for_datapath hv2 $lr1_dp_key no
> > > +check_offlows_for_datapath hv2 $sw2_dp_key no
> > > +check_offlows_for_datapath hv2 $lr2_dp_key no
> > > +
> > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr1, type: router
> > > +Datapath: lr2, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw1, type: switch
> > > +Datapath: sw2, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AS_BOX([Delete the VIF for sw0-port2 in hv2])
> > > +
> > > +# Presently when a port binding is released we are not
> > > +# deleting its datapath from the local_datapaths if it
> > > +# is not relevant anymore.
> > > +
> > > +as hv2 ovs-vsctl del-port hv2-vif2
> > > +check ovn-nbctl --wait=hv sync
> > > +check_column "false" Port_Binding up logical_port=sw0-port2
> > > +
> > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +# hv2 would still have public, sw0 and lr0 in its local datapaths.
> > > +# Next recompute should delete these datapaths.
> > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr1, type: router
> > > +Datapath: lr2, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw1, type: switch
> > > +Datapath: sw2, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +# Trigger a recompute
> > > +AS_BOX([Trigger a recompute in hv2])
> > > +check as hv2 ovn-appctl inc-engine/recompute
> > > +
> > > +check_offlows_for_datapath hv2 $sw0_dp_key no
> > > +check_offlows_for_datapath hv2 $lr0_dp_key no
> > > +check_offlows_for_datapath hv2 $public_dp_key no
> > > +check_offlows_for_datapath hv2 $sw1_dp_key no
> > > +check_offlows_for_datapath hv2 $lr1_dp_key no
> > > +check_offlows_for_datapath hv2 $sw2_dp_key no
> > > +check_offlows_for_datapath hv2 $lr2_dp_key no
> > > +
> > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr1, type: router
> > > +Datapath: lr2, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw1, type: switch
> > > +Datapath: sw2, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AS_BOX([Disconnect sw2 from lr2])
> > > +
> > > +check ovn-nbctl --wait=hv lsp-set-options sw2-lr2 router-port=lr2-sw2xxx
> > > +check_offlows_for_datapath hv1 $sw0_dp_key yes
> > > +check_offlows_for_datapath hv1 $lr0_dp_key yes
> > > +check_offlows_for_datapath hv1 $public_dp_key yes
> > > +check_offlows_for_datapath hv1 $sw1_dp_key no
> > > +check_offlows_for_datapath hv1 $lr1_dp_key no
> > > +check_offlows_for_datapath hv1 $sw2_dp_key no
> > > +check_offlows_for_datapath hv1 $lr2_dp_key no
> > > +
> > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr1, type: router
> > > +Datapath: lr2, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw1, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AS_BOX([Reconnect sw2 to lr2 again])
> > > +
> > > +check ovn-nbctl --wait=hv lsp-set-options sw2-lr2 router-port=lr2-sw2
> > > +check_offlows_for_datapath hv1 $sw0_dp_key yes
> > > +check_offlows_for_datapath hv1 $lr0_dp_key yes
> > > +check_offlows_for_datapath hv1 $public_dp_key yes
> > > +check_offlows_for_datapath hv1 $sw1_dp_key no
> > > +check_offlows_for_datapath hv1 $lr1_dp_key no
> > > +check_offlows_for_datapath hv1 $sw2_dp_key no
> > > +check_offlows_for_datapath hv1 $lr2_dp_key no
> > > +
> > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr1, type: router
> > > +Datapath: lr2, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw1, type: switch
> > > +Datapath: sw2, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AS_BOX([Create a VIF on gw2 for sw1-port1])
> > > +
> > > +as gw2
> > > +ovs-vsctl -- add-port br-int gw2-vif2 -- \
> > > +    set interface gw2-vif2 external-ids:iface-id=sw1-port1 \
> > > +    options:tx_pcap=gw2/vif2-tx.pcap \
> > > +    options:rxq_pcap=gw2/vif2-rx.pcap \
> > > +    ofport-request=2
> > > +
> > > +wait_for_ports_up sw1-port1
> > > +
> > > +check_offlows_for_datapath gw2 $sw0_dp_key no
> > > +check_offlows_for_datapath gw2 $lr0_dp_key no
> > > +check_offlows_for_datapath gw2 $public_dp_key yes
> > > +check_offlows_for_datapath gw2 $sw1_dp_key yes
> > > +check_offlows_for_datapath gw2 $lr1_dp_key yes
> > > +check_offlows_for_datapath gw2 $sw2_dp_key yes
> > > +check_offlows_for_datapath gw2 $lr2_dp_key yes
> > > +
> > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr1, type: router
> > > +Datapath: lr2, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw1, type: switch
> > > +Datapath: sw2, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AS_BOX([Delete the VIF for sw1-port1 in gw2])
> > > +
> > > +as gw2 ovs-vsctl del-port gw2-vif2
> > > +check ovn-nbctl --wait=hv sync
> > > +check_column "false" Port_Binding up logical_port=sw1-port1
> > > +
> > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +# hv2 would still have public in its local datapaths.  Next recompute should
> > > +# delete this datapath from the local datapaths.
> > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr1, type: router
> > > +Datapath: lr2, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw1, type: switch
> > > +Datapath: sw2, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AS_BOX([Create a logical port for public and bind it on hv2])
> > > +# hv2 will only have public in its local datapaths.
> > > +check ovn-nbctl lsp-add public public-p1
> > > +
> > > +as hv2
> > > +ovs-vsctl -- add-port br-int hv2-vif3 -- \
> > > +    set interface hv2-vif3 external-ids:iface-id=public-p1 \
> > > +    options:tx_pcap=hv2/vif3-tx.pcap \
> > > +    options:rxq_pcap=hv2/vif3-rx.pcap \
> > > +    ofport-request=2
> > > +
> > > +wait_for_ports_up public-p1
> > > +
> > > +# as hv2 ovn-appctl -t ovn-controller inc-engine/recompute
> > > +# check ovn-nbctl --wait=hv sync
> > > +
> > > +as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort
> > > +
> > > +check_offlows_for_datapath hv2 $sw0_dp_key no
> > > +check_offlows_for_datapath hv2 $lr0_dp_key no
> > > +check_offlows_for_datapath hv2 $public_dp_key yes
> > > +check_offlows_for_datapath hv2 $sw1_dp_key no
> > > +check_offlows_for_datapath hv2 $lr1_dp_key no
> > > +check_offlows_for_datapath hv2 $sw2_dp_key no
> > > +check_offlows_for_datapath hv2 $lr2_dp_key no
> > > +
> > > +AT_CHECK([as hv1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as hv2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: public, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw1 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr0, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw0, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +AT_CHECK([as gw2 ovn-appctl -t ovn-controller debug/dump-local-datapaths | sort], [0], [dnl
> > > +Datapath: lr1, type: router
> > > +Datapath: lr2, type: router
> > > +Datapath: public, type: switch
> > > +Datapath: sw1, type: switch
> > > +Datapath: sw2, type: switch
> > > +Local datapaths:
> > > +])
> > > +
> > > +OVN_CLEANUP([hv1], [hv2], [gw1], [gw2])
> > > +AT_CLEANUP
> > > +])
> > > +
> > >  OVN_FOR_EACH_NORTHD([
> > >  AT_SETUP([requested-tnl-key-recompute])
> > >  AT_KEYWORDS([requested-tnl-key-recompute])
> > > --
> > > 2.48.1
> > >
> > > _______________________________________________
> > > dev mailing list
> > > dev@openvswitch.org
> > > https://mail.openvswitch.org/mailman/listinfo/ovs-dev
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev