diff mbox series

[ovs-dev,v2] northd: Support routing over other address families.

Message ID 9093d78a815a8f1ab7ef352feb4c1b5897b68b70.1713790673.git.felix.huettner@mail.schwarz
State Superseded
Headers show
Series [ovs-dev,v2] northd: Support routing over other address families. | expand

Checks

Context Check Description
ovsrobot/apply-robot success apply and check: success
ovsrobot/github-robot-_ovn-kubernetes success github build: passed
ovsrobot/github-robot-_Build_and_Test fail github build: failed

Commit Message

Felix Huettner April 22, 2024, 1:01 p.m. UTC
In most cases IPv4 packets are routed only over other IPv4 networks and
IPv6 packets are routed only over IPv6 networks. However there is no
inherent reason for this limitation. Routing IPv4 packets over IPv6
networks just requires the router to contain a route for an IPv4 network
with an IPv6 nexthop.

This was previously prevented in OVN in ovn-nbctl and northd. By
removing these filters the forwarding will work if the mac addresses are
prepopulated.

If the mac addresses are not prepopulated we will attempt to resolve them using
the original address family of the packet and not the address family of the
nexthop. This will fail and we will not forward the packet.

This feature can for example be used by service providers to
interconnect multiple IPv4 networks of a customer without needing to
negotiate free IPv4 addresses by just using any IPv6 address.

Signed-off-by: Felix Huettner <felix.huettner@mail.schwarz>
---
v1->v2:
  - move ipv4 info to parsed_route
  - add tests for lr-route-add
  - switch tests to use fmt_pkt
  - some minor test cleanups

 NEWS                  |   4 +
 northd/northd.c       |  56 ++---
 tests/ovn-nbctl.at    |  26 ++-
 tests/ovn.at          | 511 ++++++++++++++++++++++++++++++++++++++++++
 utilities/ovn-nbctl.c |  12 +-
 5 files changed, 570 insertions(+), 39 deletions(-)


base-commit: c141d8b1961459a6e9a1834f33613d8be079310e

Comments

Numan Siddique May 7, 2024, 3:03 a.m. UTC | #1
On Mon, Apr 22, 2024 at 9:02 AM Felix Huettner via dev
<ovs-dev@openvswitch.org> wrote:
>
> In most cases IPv4 packets are routed only over other IPv4 networks and
> IPv6 packets are routed only over IPv6 networks. However there is no
> inherent reason for this limitation. Routing IPv4 packets over IPv6
> networks just requires the router to contain a route for an IPv4 network
> with an IPv6 nexthop.
>
> This was previously prevented in OVN in ovn-nbctl and northd. By
> removing these filters the forwarding will work if the mac addresses are
> prepopulated.
>
> If the mac addresses are not prepopulated we will attempt to resolve them using
> the original address family of the packet and not the address family of the
> nexthop. This will fail and we will not forward the packet.
>

Thanks for adding this feature.

If I understand correctly from the above commit message about
prepopulation,  is it expected that
CMS/admin needs to add a static mac binding entry in the OVN
Northbound 'Static_MAC_Binding' table
for this feature to work ?  Please correct me if I'm wrong.
If this is the case, then it needs to be documented properly (perhaps
in ovn-nb.xml)

The patch overall looks good to me.  Can you please add a small test
case in ovn-northd.at to check
the expected logical flows in the "lr_in_ip_routing" stage when you
have these mixed mode static routes ?
You can also check for correctness of flows in other tables too which
are generated for the static routes.

Thanks
Numan



> This feature can for example be used by service providers to
> interconnect multiple IPv4 networks of a customer without needing to
> negotiate free IPv4 addresses by just using any IPv6 address.
>
> Signed-off-by: Felix Huettner <felix.huettner@mail.schwarz>
> ---
> v1->v2:
>   - move ipv4 info to parsed_route
>   - add tests for lr-route-add
>   - switch tests to use fmt_pkt
>   - some minor test cleanups


>
>  NEWS                  |   4 +
>  northd/northd.c       |  56 ++---
>  tests/ovn-nbctl.at    |  26 ++-
>  tests/ovn.at          | 511 ++++++++++++++++++++++++++++++++++++++++++
>  utilities/ovn-nbctl.c |  12 +-
>  5 files changed, 570 insertions(+), 39 deletions(-)
>
> diff --git a/NEWS b/NEWS
> index 141f1831c..14a935c86 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -13,6 +13,10 @@ Post v24.03.0
>      "lflow-stage-to-oftable STAGE_NAME" that converts stage name into OpenFlow
>      table id.
>    - Rename the ovs-sandbox script to ovn-sandbox.
> +  - Allow Static Routes where the address families of ip_prefix and nexthop
> +    diverge (e.g. IPv4 packets over IPv6 links). This is currently limited to
> +    nexthops that have their mac addresses prepopulated (so
> +    dynamic_neigh_routers must be false).
>
>  OVN v24.03.0 - 01 Mar 2024
>  --------------------------
> diff --git a/northd/northd.c b/northd/northd.c
> index 331d9c267..e377d3e18 100644
> --- a/northd/northd.c
> +++ b/northd/northd.c
> @@ -10194,6 +10194,8 @@ struct parsed_route {
>      const struct nbrec_logical_router_static_route *route;
>      bool ecmp_symmetric_reply;
>      bool is_discard_route;
> +    bool is_ipv4_prefix;
> +    bool is_ipv4_nexthop;
>  };
>
>  static uint32_t
> @@ -10219,6 +10221,7 @@ parsed_routes_add(struct ovn_datapath *od, const struct hmap *lr_ports,
>      /* Verify that the next hop is an IP address with an all-ones mask. */
>      struct in6_addr nexthop;
>      unsigned int plen;
> +    bool is_ipv4_nexthop, is_ipv4_prefix;
>      bool is_discard_route = !strcmp(route->nexthop, "discard");
>      bool valid_nexthop = route->nexthop[0] && !is_discard_route;
>      if (valid_nexthop) {
> @@ -10237,6 +10240,7 @@ parsed_routes_add(struct ovn_datapath *od, const struct hmap *lr_ports,
>                           UUID_ARGS(&route->header_.uuid));
>              return NULL;
>          }
> +        is_ipv4_nexthop = IN6_IS_ADDR_V4MAPPED(&nexthop);
>      }
>
>      /* Parse ip_prefix */
> @@ -10248,18 +10252,7 @@ parsed_routes_add(struct ovn_datapath *od, const struct hmap *lr_ports,
>                       UUID_ARGS(&route->header_.uuid));
>          return NULL;
>      }
> -
> -    /* Verify that ip_prefix and nexthop have same address familiy. */
> -    if (valid_nexthop) {
> -        if (IN6_IS_ADDR_V4MAPPED(&prefix) != IN6_IS_ADDR_V4MAPPED(&nexthop)) {
> -            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
> -            VLOG_WARN_RL(&rl, "Address family doesn't match between 'ip_prefix'"
> -                         " %s and 'nexthop' %s in static route "UUID_FMT,
> -                         route->ip_prefix, route->nexthop,
> -                         UUID_ARGS(&route->header_.uuid));
> -            return NULL;
> -        }
> -    }
> +    is_ipv4_prefix = IN6_IS_ADDR_V4MAPPED(&prefix);
>
>      /* Verify that ip_prefix and nexthop are on the same network. */
>      if (!is_discard_route &&
> @@ -10302,6 +10295,8 @@ parsed_routes_add(struct ovn_datapath *od, const struct hmap *lr_ports,
>      pr->ecmp_symmetric_reply = smap_get_bool(&route->options,
>                                               "ecmp_symmetric_reply", false);
>      pr->is_discard_route = is_discard_route;
> +    pr->is_ipv4_prefix = is_ipv4_prefix;
> +    pr->is_ipv4_nexthop = is_ipv4_nexthop;
>      ovs_list_insert(routes, &pr->list_node);
>      return pr;
>  }
> @@ -10677,7 +10672,7 @@ build_ecmp_route_flow(struct lflow_table *lflows, struct ovn_datapath *od,
>                        struct lflow_ref *lflow_ref)
>
>  {
> -    bool is_ipv4 = IN6_IS_ADDR_V4MAPPED(&eg->prefix);
> +    bool is_ipv4_prefix = IN6_IS_ADDR_V4MAPPED(&eg->prefix);
>      uint16_t priority;
>      struct ecmp_route_list_node *er;
>      struct ds route_match = DS_EMPTY_INITIALIZER;
> @@ -10686,7 +10681,8 @@ build_ecmp_route_flow(struct lflow_table *lflows, struct ovn_datapath *od,
>      int ofs = !strcmp(eg->origin, ROUTE_ORIGIN_CONNECTED) ?
>          ROUTE_PRIO_OFFSET_CONNECTED: ROUTE_PRIO_OFFSET_STATIC;
>      build_route_match(NULL, eg->route_table_id, prefix_s, eg->plen,
> -                      eg->is_src_route, is_ipv4, &route_match, &priority, ofs);
> +                      eg->is_src_route, is_ipv4_prefix, &route_match,
> +                      &priority, ofs);
>      free(prefix_s);
>
>      struct ds actions = DS_EMPTY_INITIALIZER;
> @@ -10719,7 +10715,8 @@ build_ecmp_route_flow(struct lflow_table *lflows, struct ovn_datapath *od,
>          /* Find the outgoing port. */
>          const char *lrp_addr_s = NULL;
>          struct ovn_port *out_port = NULL;
> -        if (!find_static_route_outport(od, lr_ports, route, is_ipv4,
> +        if (!find_static_route_outport(od, lr_ports, route,
> +                                       route_->is_ipv4_nexthop,
>                                         &lrp_addr_s, &out_port)) {
>              continue;
>          }
> @@ -10744,9 +10741,10 @@ build_ecmp_route_flow(struct lflow_table *lflows, struct ovn_datapath *od,
>                        "eth.src = %s; "
>                        "outport = %s; "
>                        "next;",
> -                      is_ipv4 ? REG_NEXT_HOP_IPV4 : REG_NEXT_HOP_IPV6,
> +                      route_->is_ipv4_nexthop ?
> +                          REG_NEXT_HOP_IPV4 : REG_NEXT_HOP_IPV6,
>                        route->nexthop,
> -                      is_ipv4 ? REG_SRC_IPV4 : REG_SRC_IPV6,
> +                      route_->is_ipv4_nexthop ? REG_SRC_IPV4 : REG_SRC_IPV6,
>                        lrp_addr_s,
>                        out_port->lrp_networks.ea_s,
>                        out_port->json_key);
> @@ -10766,15 +10764,15 @@ add_route(struct lflow_table *lflows, struct ovn_datapath *od,
>            const char *network_s, int plen, const char *gateway,
>            bool is_src_route, const uint32_t rtb_id,
>            const struct ovsdb_idl_row *stage_hint, bool is_discard_route,
> -          int ofs, struct lflow_ref *lflow_ref)
> +          int ofs, struct lflow_ref *lflow_ref,
> +          bool is_ipv4_prefix, bool is_ipv4_nexthop)
>  {
> -    bool is_ipv4 = strchr(network_s, '.') ? true : false;
>      struct ds match = DS_EMPTY_INITIALIZER;
>      uint16_t priority;
>      const struct ovn_port *op_inport = NULL;
>
>      /* IPv6 link-local addresses must be scoped to the local router port. */
> -    if (!is_ipv4) {
> +    if (!is_ipv4_prefix) {
>          struct in6_addr network;
>          ovs_assert(ipv6_parse(network_s, &network));
>          if (in6_is_lla(&network)) {
> @@ -10782,7 +10780,7 @@ add_route(struct lflow_table *lflows, struct ovn_datapath *od,
>          }
>      }
>      build_route_match(op_inport, rtb_id, network_s, plen, is_src_route,
> -                      is_ipv4, &match, &priority, ofs);
> +                      is_ipv4_prefix, &match, &priority, ofs);
>
>      struct ds common_actions = DS_EMPTY_INITIALIZER;
>      struct ds actions = DS_EMPTY_INITIALIZER;
> @@ -10790,11 +10788,12 @@ add_route(struct lflow_table *lflows, struct ovn_datapath *od,
>          ds_put_cstr(&actions, debug_drop_action());
>      } else {
>          ds_put_format(&common_actions, REG_ECMP_GROUP_ID" = 0; %s = ",
> -                      is_ipv4 ? REG_NEXT_HOP_IPV4 : REG_NEXT_HOP_IPV6);
> +                      is_ipv4_nexthop ? REG_NEXT_HOP_IPV4 : REG_NEXT_HOP_IPV6);
>          if (gateway && gateway[0]) {
>              ds_put_cstr(&common_actions, gateway);
>          } else {
> -            ds_put_format(&common_actions, "ip%s.dst", is_ipv4 ? "4" : "6");
> +            ds_put_format(&common_actions, "ip%s.dst",
> +                          is_ipv4_prefix ? "4" : "6");
>          }
>          ds_put_format(&common_actions, "; "
>                        "%s = %s; "
> @@ -10802,7 +10801,7 @@ add_route(struct lflow_table *lflows, struct ovn_datapath *od,
>                        "outport = %s; "
>                        "flags.loopback = 1; "
>                        "next;",
> -                      is_ipv4 ? REG_SRC_IPV4 : REG_SRC_IPV6,
> +                      is_ipv4_nexthop ? REG_SRC_IPV4 : REG_SRC_IPV6,
>                        lrp_addr_s,
>                        op->lrp_networks.ea_s,
>                        op->json_key);
> @@ -10854,7 +10853,8 @@ build_static_route_flow(struct lflow_table *lflows, struct ovn_datapath *od,
>      add_route(lflows, route_->is_discard_route ? od : out_port->od, out_port,
>                lrp_addr_s, prefix_s, route_->plen, route->nexthop,
>                route_->is_src_route, route_->route_table_id, &route->header_,
> -              route_->is_discard_route, ofs, lflow_ref);
> +              route_->is_discard_route, ofs, lflow_ref,
> +              route_->is_ipv4_prefix, route_->is_ipv4_nexthop);
>
>      free(prefix_s);
>  }
> @@ -12576,7 +12576,7 @@ build_ip_routing_flows_for_lrp(
>                    op->lrp_networks.ipv4_addrs[i].network_s,
>                    op->lrp_networks.ipv4_addrs[i].plen, NULL, false, 0,
>                    &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED,
> -                  lflow_ref);
> +                  lflow_ref, true, true);
>      }
>
>      for (int i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) {
> @@ -12584,7 +12584,7 @@ build_ip_routing_flows_for_lrp(
>                    op->lrp_networks.ipv6_addrs[i].network_s,
>                    op->lrp_networks.ipv6_addrs[i].plen, NULL, false, 0,
>                    &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED,
> -                  lflow_ref);
> +                  lflow_ref, false, false);
>      }
>  }
>
> @@ -15361,7 +15361,7 @@ build_routable_flows_for_router_port(
>                                laddrs->ipv4_addrs[k].plen, NULL, false, 0,
>                                &router_port->nbrp->header_, false,
>                                ROUTE_PRIO_OFFSET_CONNECTED,
> -                              lrp->stateful_lflow_ref);
> +                              lrp->stateful_lflow_ref, true, true);
>                  }
>              }
>          }
> diff --git a/tests/ovn-nbctl.at b/tests/ovn-nbctl.at
> index 5248e6c76..4a219ab61 100644
> --- a/tests/ovn-nbctl.at
> +++ b/tests/ovn-nbctl.at
> @@ -1757,7 +1757,7 @@ AT_CHECK([ovn-nbctl lr-route-add lr0 10.0.0.1/24 11.0.0.2])
>  AT_CHECK([ovn-nbctl lr-route-add lr0 10.0.10.0/24 lp0])
>  AT_CHECK([ovn-nbctl --bfd lr-route-add lr0 10.0.20.0/24 11.0.2.1 lp0])
>  AT_CHECK([ovn-nbctl lr-route-add lr0 10.0.10.0/24 lp1], [1], [],
> -  [ovn-nbctl: bad IPv4 nexthop argument: lp1
> +  [ovn-nbctl: bad nexthop argument: lp1
>  ])
>
>  dnl Add overlapping route with 10.0.0.1/24
> @@ -1771,13 +1771,13 @@ AT_CHECK([ovn-nbctl lr-route-add lr0 10.0.0.111/24a 11.0.0.1], [1], [],
>    [ovn-nbctl: bad prefix argument: 10.0.0.111/24a
>  ])
>  AT_CHECK([ovn-nbctl lr-route-add lr0 10.0.0.111/24 11.0.0.1a], [1], [],
> -  [ovn-nbctl: bad IPv4 nexthop argument: 11.0.0.1a
> +  [ovn-nbctl: bad nexthop argument: 11.0.0.1a
>  ])
>  AT_CHECK([ovn-nbctl lr-route-add lr0 10.0.0.111/24 11.0.0.1/24], [1], [],
> -  [ovn-nbctl: bad IPv4 nexthop argument: 11.0.0.1/24
> +  [ovn-nbctl: bad nexthop argument: 11.0.0.1/24
>  ])
>  AT_CHECK([ovn-nbctl lr-route-add lr0 2001:0db8:1::/64 2001:0db8:0:f103::1/64], [1], [],
> -  [ovn-nbctl: bad IPv6 nexthop argument: 2001:0db8:0:f103::1/64
> +  [ovn-nbctl: bad nexthop argument: 2001:0db8:0:f103::1/64
>  ])
>  AT_CHECK([ovn-nbctl --ecmp lr-route-add lr0 20.0.0.0/24 discard], [1], [],
>    [ovn-nbctl: ecmp is not valid for discard routes.
> @@ -2005,6 +2005,24 @@ check ovn-nbctl lr-route-del lr0
>  AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
>  ])
>
> +dnl Check IPv4 over v6 and IPv6 over v4 routes
> +AT_CHECK([ovn-nbctl lr-route-add lr0 10.0.0.1/24 2001:0db8:0:f103::10])
> +AT_CHECK([ovn-nbctl lr-route-add lr0 2001:0db8:0::/64 11.0.1.10])
> +
> +AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
> +IPv4 Routes
> +Route Table <main>:
> +              10.0.0.0/24       2001:db8:0:f103::10 dst-ip
> +
> +IPv6 Routes
> +Route Table <main>:
> +            2001:db8::/64                 11.0.1.10 dst-ip
> +])
> +
> +check ovn-nbctl lr-route-del lr0
> +AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
> +])
> +
>  dnl Check IPv4 routes in route table
>  check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 0.0.0.0/0 192.168.0.1
>  check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 10.0.1.1/24 11.0.1.1 lp0
> diff --git a/tests/ovn.at b/tests/ovn.at
> index dc6aafd53..bf7d5ef4b 100644
> --- a/tests/ovn.at
> +++ b/tests/ovn.at
> @@ -37772,3 +37772,514 @@ OVS_WAIT_FOR_OUTPUT([as hv1 ovs-ofctl dump-flows br-int table=0 |grep priority=1
>  OVN_CLEANUP([hv1])
>  AT_CLEANUP
>  ])
> +
> +OVN_FOR_EACH_NORTHD([
> +AT_SETUP([2 HVs, 2 LS, 1 lport/LS, 2 peer LRs, IPv4 over IPv6])
> +AT_SKIP_IF([test $HAVE_SCAPY = no])
> +ovn_start
> +
> +# Logical network:
> +# Two LRs - R1 and R2 that are connected to each other as peers in 2001:db8::/64
> +# network. R1 has a switchs ls1 (192.168.1.0/24) connected to it.
> +# R2 has ls2 (172.16.1.0/24) connected to it.
> +
> +ls1_lp1_mac="f0:00:00:01:02:03"
> +rp_ls1_mac="00:00:00:01:02:03"
> +rp_ls2_mac="00:00:00:01:02:04"
> +ls2_lp1_mac="f0:00:00:01:02:04"
> +
> +ls1_lp1_ip="192.168.1.2"
> +ls2_lp1_ip="172.16.1.2"
> +
> +check ovn-nbctl lr-add R1
> +check ovn-nbctl lr-add R2
> +
> +check ovn-nbctl ls-add ls1
> +check ovn-nbctl ls-add ls2
> +
> +# Connect ls1 to R1
> +check ovn-nbctl lrp-add R1 ls1 $rp_ls1_mac 192.168.1.1/24
> +
> +check ovn-nbctl lsp-add ls1 rp-ls1 -- set Logical_Switch_Port rp-ls1 type=router \
> +  options:router-port=ls1 addresses=\"$rp_ls1_mac\"
> +
> +# Connect ls2 to R2
> +check ovn-nbctl lrp-add R2 ls2 $rp_ls2_mac 172.16.1.1/24
> +
> +check ovn-nbctl lsp-add ls2 rp-ls2 -- set Logical_Switch_Port rp-ls2 type=router \
> +  options:router-port=ls2 addresses=\"$rp_ls2_mac\"
> +
> +# Connect R1 to R2
> +check ovn-nbctl lrp-add R1 R1_R2 00:00:00:02:03:04 2001:db8::1/64 peer=R2_R1
> +check ovn-nbctl lrp-add R2 R2_R1 00:00:00:02:03:05 2001:db8::2/64 peer=R1_R2
> +
> +AT_CHECK([ovn-nbctl lr-route-add R1 "0.0.0.0/0" 2001:db8::2])
> +AT_CHECK([ovn-nbctl lr-route-add R2 "0.0.0.0/0" 2001:db8::1])
> +
> +# Create logical port ls1-lp1 in ls1
> +check ovn-nbctl lsp-add ls1 ls1-lp1 \
> +-- lsp-set-addresses ls1-lp1 "$ls1_lp1_mac $ls1_lp1_ip"
> +
> +# Create logical port ls2-lp1 in ls2
> +check ovn-nbctl lsp-add ls2 ls2-lp1 \
> +-- lsp-set-addresses ls2-lp1 "$ls2_lp1_mac $ls2_lp1_ip"
> +
> +# Create two hypervisor and create OVS ports corresponding to logical ports.
> +net_add n1
> +
> +sim_add hv1
> +as hv1
> +check ovs-vsctl add-br br-phys
> +ovn_attach n1 br-phys 192.168.0.1
> +check ovs-vsctl -- add-port br-int hv1-vif1 -- \
> +    set interface hv1-vif1 external-ids:iface-id=ls1-lp1 \
> +    options:tx_pcap=hv1/vif1-tx.pcap \
> +    options:rxq_pcap=hv1/vif1-rx.pcap \
> +    ofport-request=1
> +
> +sim_add hv2
> +as hv2
> +check ovs-vsctl add-br br-phys
> +ovn_attach n1 br-phys 192.168.0.2
> +check ovs-vsctl -- add-port br-int hv2-vif1 -- \
> +    set interface hv2-vif1 external-ids:iface-id=ls2-lp1 \
> +    options:tx_pcap=hv2/vif1-tx.pcap \
> +    options:rxq_pcap=hv2/vif1-rx.pcap \
> +    ofport-request=1
> +
> +
> +# Pre-populate the hypervisors' ARP tables so that we don't lose any
> +# packets for ARP resolution (native tunneling doesn't queue packets
> +# for ARP resolution).
> +OVN_POPULATE_ARP
> +
> +# Allow some time for ovn-northd and ovn-controller to catch up.
> +wait_for_ports_up
> +check ovn-nbctl --wait=hv sync
> +
> +# Packet to send.
> +packet=$(fmt_pkt "Ether(dst='${rp_ls1_mac}', src='${ls1_lp1_mac}')/ \
> +                        IP(src='${ls1_lp1_ip}', dst='${ls2_lp1_ip}', ttl=64)/ \
> +                        UDP(sport=53, dport=4369)")
> +check as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 "$packet"
> +
> +# Packet to Expect
> +# The TTL should be decremented by 2.
> +expected=$(fmt_pkt "Ether(dst='${ls2_lp1_mac}', src='${rp_ls2_mac}')/ \
> +                        IP(src='${ls1_lp1_ip}', dst='${ls2_lp1_ip}', ttl=62)/ \
> +                        UDP(sport=53, dport=4369)")
> +echo ${expected} > expected
> +OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
> +
> +AT_CHECK([ovn-sbctl dump-flows | grep lr_in_arp_resolve | \
> +grep "reg0 == 172.16.1.2" | wc -l], [0], [1
> +])
> +
> +# Disable the ls2-lp1 port.
> +check ovn-nbctl --wait=hv set logical_switch_port ls2-lp1 enabled=false
> +
> +AT_CHECK([ovn-sbctl dump-flows | grep lr_in_arp_resolve | \
> +grep "reg0 == 172.16.1.2" | wc -l], [0], [0
> +])
> +
> +# Send the same packet again and it should not be delivered
> +check as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 "$packet"
> +
> +# The 2nd packet sent shound not be received.
> +OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
> +
> +OVN_CLEANUP([hv1],[hv2])
> +
> +AT_CLEANUP
> +])
> +
> +OVN_FOR_EACH_NORTHD([
> +AT_SETUP([2 HVs, 2 LS, 1 lport/LS, LRs connected via LS, IPv4 over IPv6])
> +AT_SKIP_IF([test $HAVE_SCAPY = no])
> +ovn_start
> +
> +# Logical network:
> +# Two LRs - R1 and R2 that are connected to ls-transfer in 2001:db8::/64
> +# network. R1 has a switchs ls1 (192.168.1.0/24) connected to it.
> +# R2 has ls2 (172.16.1.0/24) connected to it.
> +
> +ls1_lp1_mac="f0:00:00:01:02:03"
> +rp_ls1_mac="00:00:00:01:02:03"
> +rp_ls2_mac="00:00:00:01:02:04"
> +ls2_lp1_mac="f0:00:00:01:02:04"
> +
> +ls1_lp1_ip="192.168.1.2"
> +ls2_lp1_ip="172.16.1.2"
> +
> +check ovn-nbctl lr-add R1
> +check ovn-nbctl lr-add R2
> +
> +check ovn-nbctl ls-add ls1
> +check ovn-nbctl ls-add ls2
> +check ovn-nbctl ls-add ls-transfer
> +
> +# Connect ls1 to R1
> +check ovn-nbctl lrp-add R1 ls1 $rp_ls1_mac 192.168.1.1/24
> +
> +check ovn-nbctl lsp-add ls1 rp-ls1 -- set Logical_Switch_Port rp-ls1 type=router \
> +  options:router-port=ls1 addresses=\"$rp_ls1_mac\"
> +
> +# Connect ls2 to R2
> +check ovn-nbctl lrp-add R2 ls2 $rp_ls2_mac 172.16.1.1/24
> +
> +check ovn-nbctl lsp-add ls2 rp-ls2 -- set Logical_Switch_Port rp-ls2 type=router \
> +  options:router-port=ls2 addresses=\"$rp_ls2_mac\"
> +
> +# Connect R1 to R2
> +check ovn-nbctl lrp-add R1 R1_ls-transfer 00:00:00:02:03:04 2001:db8::1/64
> +check ovn-nbctl lrp-add R2 R2_ls-transfer 00:00:00:02:03:05 2001:db8::2/64
> +
> +check ovn-nbctl lsp-add ls-transfer ls-transfer_r1 -- \
> +  set Logical_Switch_Port ls-transfer_r1 type=router \
> +  options:router-port=R1_ls-transfer addresses=\"router\"
> +check ovn-nbctl lsp-add ls-transfer ls-transfer_r2 -- \
> +  set Logical_Switch_Port ls-transfer_r2 type=router \
> +  options:router-port=R2_ls-transfer addresses=\"router\"
> +
> +AT_CHECK([ovn-nbctl lr-route-add R1 "0.0.0.0/0" 2001:db8::2])
> +AT_CHECK([ovn-nbctl lr-route-add R2 "0.0.0.0/0" 2001:db8::1])
> +
> +# Create logical port ls1-lp1 in ls1
> +check ovn-nbctl lsp-add ls1 ls1-lp1 \
> +-- lsp-set-addresses ls1-lp1 "$ls1_lp1_mac $ls1_lp1_ip"
> +
> +# Create logical port ls2-lp1 in ls2
> +check ovn-nbctl lsp-add ls2 ls2-lp1 \
> +-- lsp-set-addresses ls2-lp1 "$ls2_lp1_mac $ls2_lp1_ip"
> +
> +# Create two hypervisor and create OVS ports corresponding to logical ports.
> +net_add n1
> +
> +sim_add hv1
> +as hv1
> +check ovs-vsctl add-br br-phys
> +ovn_attach n1 br-phys 192.168.0.1
> +check ovs-vsctl -- add-port br-int hv1-vif1 -- \
> +    set interface hv1-vif1 external-ids:iface-id=ls1-lp1 \
> +    options:tx_pcap=hv1/vif1-tx.pcap \
> +    options:rxq_pcap=hv1/vif1-rx.pcap \
> +    ofport-request=1
> +
> +sim_add hv2
> +as hv2
> +check ovs-vsctl add-br br-phys
> +ovn_attach n1 br-phys 192.168.0.2
> +check ovs-vsctl -- add-port br-int hv2-vif1 -- \
> +    set interface hv2-vif1 external-ids:iface-id=ls2-lp1 \
> +    options:tx_pcap=hv2/vif1-tx.pcap \
> +    options:rxq_pcap=hv2/vif1-rx.pcap \
> +    ofport-request=1
> +
> +
> +# Pre-populate the hypervisors' ARP tables so that we don't lose any
> +# packets for ARP resolution (native tunneling doesn't queue packets
> +# for ARP resolution).
> +OVN_POPULATE_ARP
> +
> +# Allow some time for ovn-northd and ovn-controller to catch up.
> +wait_for_ports_up
> +check ovn-nbctl --wait=hv sync
> +
> +# Packet to send.
> +packet=$(fmt_pkt "Ether(dst='${rp_ls1_mac}', src='${ls1_lp1_mac}')/ \
> +                        IP(src='${ls1_lp1_ip}', dst='${ls2_lp1_ip}', ttl=64)/ \
> +                        UDP(sport=53, dport=4369)")
> +check as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 "$packet"
> +
> +# Packet to Expect
> +# The TTL should be decremented by 2.
> +expected=$(fmt_pkt "Ether(dst='${ls2_lp1_mac}', src='${rp_ls2_mac}')/ \
> +                        IP(src='${ls1_lp1_ip}', dst='${ls2_lp1_ip}', ttl=62)/ \
> +                        UDP(sport=53, dport=4369)")
> +echo ${expected} > expected
> +OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
> +
> +AT_CHECK([ovn-sbctl dump-flows | grep lr_in_arp_resolve | \
> +grep "reg0 == 172.16.1.2" | wc -l], [0], [1
> +])
> +
> +# Disable the ls2-lp1 port.
> +check ovn-nbctl --wait=hv set logical_switch_port ls2-lp1 enabled=false
> +
> +AT_CHECK([ovn-sbctl dump-flows | grep lr_in_arp_resolve | \
> +grep "reg0 == 172.16.1.2" | wc -l], [0], [0
> +])
> +
> +# Send the same packet again and it should not be delivered
> +check as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 "$packet"
> +
> +# The 2nd packet sent shound not be received.
> +OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
> +
> +OVN_CLEANUP([hv1],[hv2])
> +
> +AT_CLEANUP
> +])
> +
> +OVN_FOR_EACH_NORTHD([
> +AT_SETUP([2 HVs, 2 LS, 1 lport/LS, LRs connected via LS, IPv4 over IPv6, ECMP])
> +AT_SKIP_IF([test $HAVE_SCAPY = no])
> +ovn_start
> +
> +# Logical network:
> +# Two LRs - R1 and R2 that are connected to ls-transfer1 and lr-transfer2 in
> +# 2001:db8:1::/64 and 2001:db8:2::/64
> +# network. R1 has a switchs ls1 (192.168.1.0/24) connected to it.
> +# R2 has ls2 (172.16.1.0/24) connected to it.
> +
> +ls1_lp1_mac="f0:00:00:01:02:03"
> +rp_ls1_mac="00:00:00:01:02:03"
> +rp_ls2_mac="00:00:00:01:02:04"
> +ls2_lp1_mac="f0:00:00:01:02:04"
> +
> +ls1_lp1_ip="192.168.1.2"
> +ls2_lp1_ip="172.16.1.2"
> +
> +check ovn-nbctl lr-add R1
> +check ovn-nbctl lr-add R2
> +
> +check ovn-nbctl ls-add ls1
> +check ovn-nbctl ls-add ls2
> +check ovn-nbctl ls-add ls-transfer1
> +check ovn-nbctl ls-add ls-transfer2
> +
> +# Connect ls1 to R1
> +check ovn-nbctl lrp-add R1 ls1 $rp_ls1_mac 192.168.1.1/24
> +
> +check ovn-nbctl lsp-add ls1 rp-ls1 -- set Logical_Switch_Port rp-ls1 type=router \
> +  options:router-port=ls1 addresses=\"$rp_ls1_mac\"
> +
> +# Connect ls2 to R2
> +check ovn-nbctl lrp-add R2 ls2 $rp_ls2_mac 172.16.1.1/24
> +
> +check ovn-nbctl lsp-add ls2 rp-ls2 -- set Logical_Switch_Port rp-ls2 type=router \
> +  options:router-port=ls2 addresses=\"$rp_ls2_mac\"
> +
> +# Connect R1 to R2 (ls-transfer1)
> +check ovn-nbctl lrp-add R1 R1_ls-transfer1 00:00:00:02:03:04 2001:db8:1::1/64
> +check ovn-nbctl lrp-add R2 R2_ls-transfer1 00:00:00:02:03:05 2001:db8:1::2/64
> +
> +check ovn-nbctl lsp-add ls-transfer1 ls-transfer1_r1 -- \
> +  set Logical_Switch_Port ls-transfer1_r1 type=router \
> +  options:router-port=R1_ls-transfer1 addresses=\"router\"
> +check ovn-nbctl lsp-add ls-transfer1 ls-transfer1_r2 -- \
> +  set Logical_Switch_Port ls-transfer1_r2 type=router \
> +  options:router-port=R2_ls-transfer1 addresses=\"router\"
> +
> +# Connect R1 to R2 (ls-transfer2)
> +check ovn-nbctl lrp-add R1 R1_ls-transfer2 00:00:00:02:03:14 2001:db8:2::1/64
> +check ovn-nbctl lrp-add R2 R2_ls-transfer2 00:00:00:02:03:15 2001:db8:2::2/64
> +
> +check ovn-nbctl lsp-add ls-transfer2 ls-transfer2_r1 -- \
> +  set Logical_Switch_Port ls-transfer2_r1 type=router \
> +  options:router-port=R1_ls-transfer2 addresses=\"router\"
> +check ovn-nbctl lsp-add ls-transfer2 ls-transfer2_r2 -- \
> +  set Logical_Switch_Port ls-transfer2_r2 type=router \
> +  options:router-port=R2_ls-transfer2 addresses=\"router\"
> +
> +AT_CHECK([ovn-nbctl lr-route-add R1 "0.0.0.0/0" 2001:db8:1::2])
> +AT_CHECK([ovn-nbctl --ecmp lr-route-add R1 "0.0.0.0/0" 2001:db8:2::2])
> +AT_CHECK([ovn-nbctl lr-route-add R2 "0.0.0.0/0" 2001:db8:1::1])
> +AT_CHECK([ovn-nbctl --ecmp lr-route-add R2 "0.0.0.0/0" 2001:db8:2::1])
> +
> +# Create logical port ls1-lp1 in ls1
> +check ovn-nbctl lsp-add ls1 ls1-lp1 \
> +-- lsp-set-addresses ls1-lp1 "$ls1_lp1_mac $ls1_lp1_ip"
> +
> +# Create logical port ls2-lp1 in ls2
> +check ovn-nbctl lsp-add ls2 ls2-lp1 \
> +-- lsp-set-addresses ls2-lp1 "$ls2_lp1_mac $ls2_lp1_ip"
> +
> +# Create two hypervisor and create OVS ports corresponding to logical ports.
> +net_add n1
> +
> +sim_add hv1
> +as hv1
> +check ovs-vsctl add-br br-phys
> +ovn_attach n1 br-phys 192.168.0.1
> +check ovs-vsctl -- add-port br-int hv1-vif1 -- \
> +    set interface hv1-vif1 external-ids:iface-id=ls1-lp1 \
> +    options:tx_pcap=hv1/vif1-tx.pcap \
> +    options:rxq_pcap=hv1/vif1-rx.pcap \
> +    ofport-request=1
> +
> +sim_add hv2
> +as hv2
> +check ovs-vsctl add-br br-phys
> +ovn_attach n1 br-phys 192.168.0.2
> +check ovs-vsctl -- add-port br-int hv2-vif1 -- \
> +    set interface hv2-vif1 external-ids:iface-id=ls2-lp1 \
> +    options:tx_pcap=hv2/vif1-tx.pcap \
> +    options:rxq_pcap=hv2/vif1-rx.pcap \
> +    ofport-request=1
> +
> +
> +# Pre-populate the hypervisors' ARP tables so that we don't lose any
> +# packets for ARP resolution (native tunneling doesn't queue packets
> +# for ARP resolution).
> +OVN_POPULATE_ARP
> +
> +# Allow some time for ovn-northd and ovn-controller to catch up.
> +wait_for_ports_up
> +check ovn-nbctl --wait=hv sync
> +
> +# Packet to send.
> +packet=$(fmt_pkt "Ether(dst='${rp_ls1_mac}', src='${ls1_lp1_mac}')/ \
> +                        IP(src='${ls1_lp1_ip}', dst='${ls2_lp1_ip}', ttl=64)/ \
> +                        UDP(sport=53, dport=4369)")
> +check as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 "$packet"
> +
> +# Packet to Expect
> +# The TTL should be decremented by 2.
> +expected=$(fmt_pkt "Ether(dst='${ls2_lp1_mac}', src='${rp_ls2_mac}')/ \
> +                        IP(src='${ls1_lp1_ip}', dst='${ls2_lp1_ip}', ttl=62)/ \
> +                        UDP(sport=53, dport=4369)")
> +echo ${expected} > expected
> +OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
> +
> +AT_CHECK([ovn-sbctl dump-flows | grep lr_in_arp_resolve | \
> +grep "reg0 == 172.16.1.2" | wc -l], [0], [1
> +])
> +
> +# Disable the ls2-lp1 port.
> +check ovn-nbctl --wait=hv set logical_switch_port ls2-lp1 enabled=false
> +
> +AT_CHECK([ovn-sbctl dump-flows | grep lr_in_arp_resolve | \
> +grep "reg0 == 172.16.1.2" | wc -l], [0], [0
> +])
> +
> +# Send the same packet again and it should not be delivered
> +check as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 "$packet"
> +
> +# The 2nd packet sent shound not be received.
> +OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
> +
> +OVN_CLEANUP([hv1],[hv2])
> +
> +AT_CLEANUP
> +])
> +
> +OVN_FOR_EACH_NORTHD([
> +AT_SETUP([2 HVs, 2 LS, 1 lport/LS, 2 peer LRs, IPv6 over IPv4])
> +AT_SKIP_IF([test $HAVE_SCAPY = no])
> +ovn_start
> +
> +# Logical network:
> +# Two LRs - R1 and R2 that are connected to each other as peers in 10.0.0.0/24
> +# network. R1 has a switchs ls1 (2001:db8:1::/64) connected to it.
> +# R2 has ls2 (2001:db8:2::/64) connected to it.
> +
> +ls1_lp1_mac="f0:00:00:01:02:03"
> +rp_ls1_mac="00:00:00:01:02:03"
> +rp_ls2_mac="00:00:00:01:02:04"
> +ls2_lp1_mac="f0:00:00:01:02:04"
> +
> +ls1_lp1_ip="2001:db8:1::2"
> +ls2_lp1_ip="2001:db8:2::2"
> +
> +check ovn-nbctl lr-add R1
> +check ovn-nbctl lr-add R2
> +
> +check ovn-nbctl ls-add ls1
> +check ovn-nbctl ls-add ls2
> +
> +# Connect ls1 to R1
> +check ovn-nbctl lrp-add R1 ls1 $rp_ls1_mac 2001:db8:1::1/64
> +
> +check ovn-nbctl lsp-add ls1 rp-ls1 -- set Logical_Switch_Port rp-ls1 type=router \
> +  options:router-port=ls1 addresses=\"$rp_ls1_mac\"
> +
> +# Connect ls2 to R2
> +check ovn-nbctl lrp-add R2 ls2 $rp_ls2_mac 2001:db8:2::1/64
> +
> +check ovn-nbctl lsp-add ls2 rp-ls2 -- set Logical_Switch_Port rp-ls2 type=router \
> +  options:router-port=ls2 addresses=\"$rp_ls2_mac\"
> +
> +# Connect R1 to R2
> +check ovn-nbctl lrp-add R1 R1_R2 00:00:00:02:03:04 10.0.0.1/24 peer=R2_R1
> +check ovn-nbctl lrp-add R2 R2_R1 00:00:00:02:03:05 10.0.0.2/24 peer=R1_R2
> +
> +AT_CHECK([ovn-nbctl lr-route-add R1 "::/0" 10.0.0.2])
> +AT_CHECK([ovn-nbctl lr-route-add R2 "::/0" 10.0.0.1])
> +
> +# Create logical port ls1-lp1 in ls1
> +check ovn-nbctl lsp-add ls1 ls1-lp1 \
> +-- lsp-set-addresses ls1-lp1 "$ls1_lp1_mac $ls1_lp1_ip"
> +
> +# Create logical port ls2-lp1 in ls2
> +check ovn-nbctl lsp-add ls2 ls2-lp1 \
> +-- lsp-set-addresses ls2-lp1 "$ls2_lp1_mac $ls2_lp1_ip"
> +
> +# Create two hypervisor and create OVS ports corresponding to logical ports.
> +net_add n1
> +
> +sim_add hv1
> +as hv1
> +check ovs-vsctl add-br br-phys
> +ovn_attach n1 br-phys 192.168.0.1
> +check ovs-vsctl -- add-port br-int hv1-vif1 -- \
> +    set interface hv1-vif1 external-ids:iface-id=ls1-lp1 \
> +    options:tx_pcap=hv1/vif1-tx.pcap \
> +    options:rxq_pcap=hv1/vif1-rx.pcap \
> +    ofport-request=1
> +
> +sim_add hv2
> +as hv2
> +check ovs-vsctl add-br br-phys
> +ovn_attach n1 br-phys 192.168.0.2
> +check ovs-vsctl -- add-port br-int hv2-vif1 -- \
> +    set interface hv2-vif1 external-ids:iface-id=ls2-lp1 \
> +    options:tx_pcap=hv2/vif1-tx.pcap \
> +    options:rxq_pcap=hv2/vif1-rx.pcap \
> +    ofport-request=1
> +
> +
> +# Pre-populate the hypervisors' ARP tables so that we don't lose any
> +# packets for ARP resolution (native tunneling doesn't queue packets
> +# for ARP resolution).
> +OVN_POPULATE_ARP
> +
> +# Allow some time for ovn-northd and ovn-controller to catch up.
> +wait_for_ports_up
> +check ovn-nbctl --wait=hv sync
> +
> +# Packet to send.
> +packet=$(fmt_pkt "Ether(dst='${rp_ls1_mac}', src='${ls1_lp1_mac}')/ \
> +                        IPv6(src='${ls1_lp1_ip}', dst='${ls2_lp1_ip}', hlim=64)/ \
> +                        UDP(sport=53, dport=4369)")
> +check as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 "$packet"
> +
> +# Packet to Expect
> +# The TTL should be decremented by 2.
> +expected=$(fmt_pkt "Ether(dst='${ls2_lp1_mac}', src='${rp_ls2_mac}')/ \
> +                        IPv6(src='${ls1_lp1_ip}', dst='${ls2_lp1_ip}', hlim=62)/ \
> +                        UDP(sport=53, dport=4369)")
> +echo ${expected} > expected
> +OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
> +
> +AT_CHECK([ovn-sbctl dump-flows | grep lr_in_arp_resolve | \
> +grep "xxreg0 == 2001:db8:2::2" | wc -l], [0], [1
> +])
> +
> +# Disable the ls2-lp1 port.
> +check ovn-nbctl --wait=hv set logical_switch_port ls2-lp1 enabled=false
> +
> +AT_CHECK([ovn-sbctl dump-flows | grep lr_in_arp_resolve | \
> +grep "xxreg0 == 2001:db8:2::2" | wc -l], [0], [0
> +])
> +
> +# Send the same packet again and it should not be delivered
> +check as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 "$packet"
> +
> +# The 2nd packet sent shound not be received.
> +OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
> +
> +OVN_CLEANUP([hv1],[hv2])
> +
> +AT_CLEANUP
> +])
> diff --git a/utilities/ovn-nbctl.c b/utilities/ovn-nbctl.c
> index 25eb86f7f..f827b2ad9 100644
> --- a/utilities/ovn-nbctl.c
> +++ b/utilities/ovn-nbctl.c
> @@ -4546,11 +4546,9 @@ nbctl_lr_route_add(struct ctl_context *ctx)
>      }
>
>      char *route_table = shash_find_data(&ctx->options, "--route-table");
> -    bool v6_prefix = false;
>      prefix = normalize_ipv4_prefix_str(ctx->argv[2]);
>      if (!prefix) {
>          prefix = normalize_ipv6_prefix_str(ctx->argv[2]);
> -        v6_prefix = true;
>      }
>      if (!prefix) {
>          ctl_error(ctx, "bad prefix argument: %s", ctx->argv[2]);
> @@ -4561,15 +4559,15 @@ nbctl_lr_route_add(struct ctl_context *ctx)
>      if (is_discard_route) {
>          next_hop = xasprintf("discard");
>      } else {
> -        next_hop = v6_prefix
> -            ? normalize_ipv6_addr_str(ctx->argv[3])
> -            : normalize_ipv4_addr_str(ctx->argv[3]);
> +        next_hop = normalize_ipv4_addr_str(ctx->argv[3]);
> +        if (!next_hop) {
> +            next_hop = normalize_ipv6_addr_str(ctx->argv[3]);
> +        }
>          if (!next_hop) {
>              /* check if it is a output port. */
>              error = lrp_by_name_or_uuid(ctx, ctx->argv[3], true, &out_lrp);
>              if (error) {
> -                ctl_error(ctx, "bad %s nexthop argument: %s",
> -                          v6_prefix ? "IPv6" : "IPv4", ctx->argv[3]);
> +                ctl_error(ctx, "bad nexthop argument: %s", ctx->argv[3]);
>                  free(error);
>                  goto cleanup;
>              }
>
> base-commit: c141d8b1961459a6e9a1834f33613d8be079310e
> --
> 2.44.0
>
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>
Felix Huettner May 7, 2024, 9:23 a.m. UTC | #2
On Mon, May 06, 2024 at 11:03:42PM -0400, Numan Siddique wrote:
> On Mon, Apr 22, 2024 at 9:02 AM Felix Huettner via dev
> <ovs-dev@openvswitch.org> wrote:
> >
> > In most cases IPv4 packets are routed only over other IPv4 networks and
> > IPv6 packets are routed only over IPv6 networks. However there is no
> > inherent reason for this limitation. Routing IPv4 packets over IPv6
> > networks just requires the router to contain a route for an IPv4 network
> > with an IPv6 nexthop.
> >
> > This was previously prevented in OVN in ovn-nbctl and northd. By
> > removing these filters the forwarding will work if the mac addresses are
> > prepopulated.
> >
> > If the mac addresses are not prepopulated we will attempt to resolve them using
> > the original address family of the packet and not the address family of the
> > nexthop. This will fail and we will not forward the packet.
> >
> 
> Thanks for adding this feature.
> 
> If I understand correctly from the above commit message about
> prepopulation,  is it expected that
> CMS/admin needs to add a static mac binding entry in the OVN
> Northbound 'Static_MAC_Binding' table
> for this feature to work ?  Please correct me if I'm wrong.
> If this is the case, then it needs to be documented properly (perhaps
> in ovn-nb.xml)

no the limitation is only that the CMS must not set
dynamic_neigh_routers to true on the Logical_Routers.

If there are other reasons that cause OVN to send ARP request to lookup
the nexthop of a route then these will have that issue as well. I am
just no aware of any such reason.

> 
> The patch overall looks good to me.  Can you please add a small test
> case in ovn-northd.at to check
> the expected logical flows in the "lr_in_ip_routing" stage when you
> have these mixed mode static routes ?
> You can also check for correctness of flows in other tables too which
> are generated for the static routes.

Yes i'll do that and send a new version.

Thanks
Felix

> 
> Thanks
> Numan
> 
> 
>
Numan Siddique May 7, 2024, 2:06 p.m. UTC | #3
On Tue, May 7, 2024 at 5:24 AM Felix Huettner via dev
<ovs-dev@openvswitch.org> wrote:
>
> On Mon, May 06, 2024 at 11:03:42PM -0400, Numan Siddique wrote:
> > On Mon, Apr 22, 2024 at 9:02 AM Felix Huettner via dev
> > <ovs-dev@openvswitch.org> wrote:
> > >
> > > In most cases IPv4 packets are routed only over other IPv4 networks and
> > > IPv6 packets are routed only over IPv6 networks. However there is no
> > > inherent reason for this limitation. Routing IPv4 packets over IPv6
> > > networks just requires the router to contain a route for an IPv4 network
> > > with an IPv6 nexthop.
> > >
> > > This was previously prevented in OVN in ovn-nbctl and northd. By
> > > removing these filters the forwarding will work if the mac addresses are
> > > prepopulated.
> > >
> > > If the mac addresses are not prepopulated we will attempt to resolve them using
> > > the original address family of the packet and not the address family of the
> > > nexthop. This will fail and we will not forward the packet.
> > >
> >
> > Thanks for adding this feature.
> >
> > If I understand correctly from the above commit message about
> > prepopulation,  is it expected that
> > CMS/admin needs to add a static mac binding entry in the OVN
> > Northbound 'Static_MAC_Binding' table
> > for this feature to work ?  Please correct me if I'm wrong.
> > If this is the case, then it needs to be documented properly (perhaps
> > in ovn-nb.xml)
>
> no the limitation is only that the CMS must not set
> dynamic_neigh_routers to true on the Logical_Routers.
>
> If there are other reasons that cause OVN to send ARP request to lookup
> the nexthop of a route then these will have that issue as well. I am
> just no aware of any such reason.

Thanks.  I'm still not clear what happens if OVN doesn't know the
nexthop mac address.
What happens in that case ? Normally, OVN would generate ARP request
for the next hop IP and
learn the mac.  What happens in the mix route case ?

Numan

>
> >
> > The patch overall looks good to me.  Can you please add a small test
> > case in ovn-northd.at to check
> > the expected logical flows in the "lr_in_ip_routing" stage when you
> > have these mixed mode static routes ?
> > You can also check for correctness of flows in other tables too which
> > are generated for the static routes.
>
> Yes i'll do that and send a new version.
>
> Thanks
> Felix
>
> >
> > Thanks
> > Numan
> >
> >
> >
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Felix Huettner May 7, 2024, 3:09 p.m. UTC | #4
On Tue, May 07, 2024 at 10:06:07AM -0400, Numan Siddique wrote:
> On Tue, May 7, 2024 at 5:24 AM Felix Huettner via dev
> <ovs-dev@openvswitch.org> wrote:
> >
> > On Mon, May 06, 2024 at 11:03:42PM -0400, Numan Siddique wrote:
> > > On Mon, Apr 22, 2024 at 9:02 AM Felix Huettner via dev
> > > <ovs-dev@openvswitch.org> wrote:
> > > >
> > > > In most cases IPv4 packets are routed only over other IPv4 networks and
> > > > IPv6 packets are routed only over IPv6 networks. However there is no
> > > > inherent reason for this limitation. Routing IPv4 packets over IPv6
> > > > networks just requires the router to contain a route for an IPv4 network
> > > > with an IPv6 nexthop.
> > > >
> > > > This was previously prevented in OVN in ovn-nbctl and northd. By
> > > > removing these filters the forwarding will work if the mac addresses are
> > > > prepopulated.
> > > >
> > > > If the mac addresses are not prepopulated we will attempt to resolve them using
> > > > the original address family of the packet and not the address family of the
> > > > nexthop. This will fail and we will not forward the packet.
> > > >
> > >
> > > Thanks for adding this feature.
> > >
> > > If I understand correctly from the above commit message about
> > > prepopulation,  is it expected that
> > > CMS/admin needs to add a static mac binding entry in the OVN
> > > Northbound 'Static_MAC_Binding' table
> > > for this feature to work ?  Please correct me if I'm wrong.
> > > If this is the case, then it needs to be documented properly (perhaps
> > > in ovn-nb.xml)
> >
> > no the limitation is only that the CMS must not set
> > dynamic_neigh_routers to true on the Logical_Routers.
> >
> > If there are other reasons that cause OVN to send ARP request to lookup
> > the nexthop of a route then these will have that issue as well. I am
> > just no aware of any such reason.
> 
> Thanks.  I'm still not clear what happens if OVN doesn't know the
> nexthop mac address.
> What happens in that case ? Normally, OVN would generate ARP request
> for the next hop IP and
> learn the mac.  What happens in the mix route case ?
> 
> Numan

So lets assume you run an ipv4 prefix and route that over an ipv6 network.
If you then reach the Logical_Router you will reach the arp action in
S_ROUTER_IN_ARP_REQUEST (build_arp_request_flows_for_lrouter).

This will be handled by the node executing that action as a
packet-in to pinctrl_handle_arp which is here actually inappropriate.
We would need to reach pinctrl_handle_nd_na as we need to send a
neighbor discovery packet.

It could be changed in northd.c when using an additional register bit
to store the nexthop address family and then using that bit to figure
out what address family we need to lookup against, instead of relying on
the packet address family.

However if we end up in pinctrl_handle_nd_na we seem to reuse the
existing ipv4 packet and patch the Neighbor Discovery packet over it.
I thought this causes the lookup then to fail, as the address families
missmatch.
However i am no longer sure of that explanation after writing this :)

Thanks
Felix
Numan Siddique May 7, 2024, 4:47 p.m. UTC | #5
On Tue, May 7, 2024 at 11:10 AM Felix Huettner via dev
<ovs-dev@openvswitch.org> wrote:
>
> On Tue, May 07, 2024 at 10:06:07AM -0400, Numan Siddique wrote:
> > On Tue, May 7, 2024 at 5:24 AM Felix Huettner via dev
> > <ovs-dev@openvswitch.org> wrote:
> > >
> > > On Mon, May 06, 2024 at 11:03:42PM -0400, Numan Siddique wrote:
> > > > On Mon, Apr 22, 2024 at 9:02 AM Felix Huettner via dev
> > > > <ovs-dev@openvswitch.org> wrote:
> > > > >
> > > > > In most cases IPv4 packets are routed only over other IPv4 networks and
> > > > > IPv6 packets are routed only over IPv6 networks. However there is no
> > > > > inherent reason for this limitation. Routing IPv4 packets over IPv6
> > > > > networks just requires the router to contain a route for an IPv4 network
> > > > > with an IPv6 nexthop.
> > > > >
> > > > > This was previously prevented in OVN in ovn-nbctl and northd. By
> > > > > removing these filters the forwarding will work if the mac addresses are
> > > > > prepopulated.
> > > > >
> > > > > If the mac addresses are not prepopulated we will attempt to resolve them using
> > > > > the original address family of the packet and not the address family of the
> > > > > nexthop. This will fail and we will not forward the packet.
> > > > >
> > > >
> > > > Thanks for adding this feature.
> > > >
> > > > If I understand correctly from the above commit message about
> > > > prepopulation,  is it expected that
> > > > CMS/admin needs to add a static mac binding entry in the OVN
> > > > Northbound 'Static_MAC_Binding' table
> > > > for this feature to work ?  Please correct me if I'm wrong.
> > > > If this is the case, then it needs to be documented properly (perhaps
> > > > in ovn-nb.xml)
> > >
> > > no the limitation is only that the CMS must not set
> > > dynamic_neigh_routers to true on the Logical_Routers.
> > >
> > > If there are other reasons that cause OVN to send ARP request to lookup
> > > the nexthop of a route then these will have that issue as well. I am
> > > just no aware of any such reason.
> >
> > Thanks.  I'm still not clear what happens if OVN doesn't know the
> > nexthop mac address.
> > What happens in that case ? Normally, OVN would generate ARP request
> > for the next hop IP and
> > learn the mac.  What happens in the mix route case ?
> >
> > Numan
>
> So lets assume you run an ipv4 prefix and route that over an ipv6 network.
> If you then reach the Logical_Router you will reach the arp action in
> S_ROUTER_IN_ARP_REQUEST (build_arp_request_flows_for_lrouter).
>
> This will be handled by the node executing that action as a
> packet-in to pinctrl_handle_arp which is here actually inappropriate.
> We would need to reach pinctrl_handle_nd_na as we need to send a
> neighbor discovery packet.
>
> It could be changed in northd.c when using an additional register bit
> to store the nexthop address family and then using that bit to figure
> out what address family we need to lookup against, instead of relying on
> the packet address family.
>
> However if we end up in pinctrl_handle_nd_na we seem to reuse the
> existing ipv4 packet and patch the Neighbor Discovery packet over it.
> I thought this causes the lookup then to fail, as the address families
> missmatch.
> However i am no longer sure of that explanation after writing this :)

Let's say we add this static route -  "prefix = 172.168.0.0/24,
nexthop = aef0::4"
and lets assume OVN doesn't know the next hop "aef0::4" mac address.
So with your patch, the original packet will be dropped because
pinctrl.c is not handling it properly.

I see 2 approaches here :
1.  Fix it in ovn-northd/pinctrl so that for IPv6 next hop, neigh
solicit packet is generated and for IPv4 next hop, ARP request packet
is generated.
2.  CMS should add a static mac binding entry in the OVN Northbound db
for the next hop for this scenario to work properly.
     Eg.  ovn-nbctl static-mac-binding-add lr0-public aef0::5 10:14:00:00:00:05
     In this case, ovn-controller(s) will add an openflow  like below

---
 ovs-ofctl dump-flows br-int table=66
 cookie=0xa4249f73, duration=470.024s, table=66, n_packets=0,
n_bytes=0, priority=50,reg0=0xaef00000,reg1=0,reg2=0,reg3=0x5,reg15=0x3,metadata=0x3
actions=mod_dl_dst:10:14:00:00:00:05,load:0x1->NXM_NX_REG10[6]
----

ovn-northd adds below logical flows in the router ingress pipeline
stage - "lr_in_arp_resolve".  These logical flows are hit if OVN can't
resolve the next hop mac address

  table=20(lr_in_arp_resolve  ), priority=1    , match=(ip4),
action=(get_arp(outport, reg0); next;)
  table=20(lr_in_arp_resolve  ), priority=1    , match=(ip6),
action=(get_nd(outport, xxreg0); next;)

The OVN actions get_arp and get_nd gets translated to OVS actions -
resubmit(,66).
The problem is that for IPv4 packets, reg0 is used and for IPv6
packetsm xxreg0 is used.  But with your patch to support mixed static
routes,  it will not work.

For both the approaches (whichever we chose to take),  we need to
solve this problem.  Maybe we can use a register bit to indicate if
nexhop is IPv4 addr or IPv6 addr
in the "lr_in_ip_routing" stage and depending on that we can have
below logical flows,

 table=20(lr_in_arp_resolve  ), priority=1    , match=(reg10[1] == 0),
action=(get_arp(outport, reg0); next;)
  table=20(lr_in_arp_resolve  ), priority=1    , match=(eg10[1] == 1),
action=(get_nd(outport, xxreg0); next;)

I think your next version should address this, otherwise the feature
will be incomplete.

If we go with approach (1),  we can indicate pinctrl to generate ARP
or NS depending on a register bit.

Thanks
Numan

>
> Thanks
> Felix
>
> _______________________________________________
> dev mailing list
> dev@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Felix Huettner May 8, 2024, 5:44 a.m. UTC | #6
> 
> Let's say we add this static route -  "prefix = 172.168.0.0/24,
> nexthop = aef0::4"
> and lets assume OVN doesn't know the next hop "aef0::4" mac address.
> So with your patch, the original packet will be dropped because
> pinctrl.c is not handling it properly.
> 
> I see 2 approaches here :
> 1.  Fix it in ovn-northd/pinctrl so that for IPv6 next hop, neigh
> solicit packet is generated and for IPv4 next hop, ARP request packet
> is generated.
> 2.  CMS should add a static mac binding entry in the OVN Northbound db
> for the next hop for this scenario to work properly.
>      Eg.  ovn-nbctl static-mac-binding-add lr0-public aef0::5 10:14:00:00:00:05
>      In this case, ovn-controller(s) will add an openflow  like below
> 
> ---
>  ovs-ofctl dump-flows br-int table=66
>  cookie=0xa4249f73, duration=470.024s, table=66, n_packets=0,
> n_bytes=0, priority=50,reg0=0xaef00000,reg1=0,reg2=0,reg3=0x5,reg15=0x3,metadata=0x3
> actions=mod_dl_dst:10:14:00:00:00:05,load:0x1->NXM_NX_REG10[6]
> ----
> 
> ovn-northd adds below logical flows in the router ingress pipeline
> stage - "lr_in_arp_resolve".  These logical flows are hit if OVN can't
> resolve the next hop mac address
> 
>   table=20(lr_in_arp_resolve  ), priority=1    , match=(ip4),
> action=(get_arp(outport, reg0); next;)
>   table=20(lr_in_arp_resolve  ), priority=1    , match=(ip6),
> action=(get_nd(outport, xxreg0); next;)
> 
> The OVN actions get_arp and get_nd gets translated to OVS actions -
> resubmit(,66).
> The problem is that for IPv4 packets, reg0 is used and for IPv6
> packetsm xxreg0 is used.  But with your patch to support mixed static
> routes,  it will not work.
> 
> For both the approaches (whichever we chose to take),  we need to
> solve this problem.  Maybe we can use a register bit to indicate if
> nexhop is IPv4 addr or IPv6 addr
> in the "lr_in_ip_routing" stage and depending on that we can have
> below logical flows,
> 
>  table=20(lr_in_arp_resolve  ), priority=1    , match=(reg10[1] == 0),
> action=(get_arp(outport, reg0); next;)
>   table=20(lr_in_arp_resolve  ), priority=1    , match=(eg10[1] == 1),
> action=(get_nd(outport, xxreg0); next;)
> 
> I think your next version should address this, otherwise the feature
> will be incomplete.
> 
> If we go with approach (1),  we can indicate pinctrl to generate ARP
> or NS depending on a register bit.
> 
> Thanks
> Numan
> 

Sounds good, will do that in the coming weeks.

Thank you
Felix
diff mbox series

Patch

diff --git a/NEWS b/NEWS
index 141f1831c..14a935c86 100644
--- a/NEWS
+++ b/NEWS
@@ -13,6 +13,10 @@  Post v24.03.0
     "lflow-stage-to-oftable STAGE_NAME" that converts stage name into OpenFlow
     table id.
   - Rename the ovs-sandbox script to ovn-sandbox.
+  - Allow Static Routes where the address families of ip_prefix and nexthop
+    diverge (e.g. IPv4 packets over IPv6 links). This is currently limited to
+    nexthops that have their mac addresses prepopulated (so
+    dynamic_neigh_routers must be false).
 
 OVN v24.03.0 - 01 Mar 2024
 --------------------------
diff --git a/northd/northd.c b/northd/northd.c
index 331d9c267..e377d3e18 100644
--- a/northd/northd.c
+++ b/northd/northd.c
@@ -10194,6 +10194,8 @@  struct parsed_route {
     const struct nbrec_logical_router_static_route *route;
     bool ecmp_symmetric_reply;
     bool is_discard_route;
+    bool is_ipv4_prefix;
+    bool is_ipv4_nexthop;
 };
 
 static uint32_t
@@ -10219,6 +10221,7 @@  parsed_routes_add(struct ovn_datapath *od, const struct hmap *lr_ports,
     /* Verify that the next hop is an IP address with an all-ones mask. */
     struct in6_addr nexthop;
     unsigned int plen;
+    bool is_ipv4_nexthop, is_ipv4_prefix;
     bool is_discard_route = !strcmp(route->nexthop, "discard");
     bool valid_nexthop = route->nexthop[0] && !is_discard_route;
     if (valid_nexthop) {
@@ -10237,6 +10240,7 @@  parsed_routes_add(struct ovn_datapath *od, const struct hmap *lr_ports,
                          UUID_ARGS(&route->header_.uuid));
             return NULL;
         }
+        is_ipv4_nexthop = IN6_IS_ADDR_V4MAPPED(&nexthop);
     }
 
     /* Parse ip_prefix */
@@ -10248,18 +10252,7 @@  parsed_routes_add(struct ovn_datapath *od, const struct hmap *lr_ports,
                      UUID_ARGS(&route->header_.uuid));
         return NULL;
     }
-
-    /* Verify that ip_prefix and nexthop have same address familiy. */
-    if (valid_nexthop) {
-        if (IN6_IS_ADDR_V4MAPPED(&prefix) != IN6_IS_ADDR_V4MAPPED(&nexthop)) {
-            static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1);
-            VLOG_WARN_RL(&rl, "Address family doesn't match between 'ip_prefix'"
-                         " %s and 'nexthop' %s in static route "UUID_FMT,
-                         route->ip_prefix, route->nexthop,
-                         UUID_ARGS(&route->header_.uuid));
-            return NULL;
-        }
-    }
+    is_ipv4_prefix = IN6_IS_ADDR_V4MAPPED(&prefix);
 
     /* Verify that ip_prefix and nexthop are on the same network. */
     if (!is_discard_route &&
@@ -10302,6 +10295,8 @@  parsed_routes_add(struct ovn_datapath *od, const struct hmap *lr_ports,
     pr->ecmp_symmetric_reply = smap_get_bool(&route->options,
                                              "ecmp_symmetric_reply", false);
     pr->is_discard_route = is_discard_route;
+    pr->is_ipv4_prefix = is_ipv4_prefix;
+    pr->is_ipv4_nexthop = is_ipv4_nexthop;
     ovs_list_insert(routes, &pr->list_node);
     return pr;
 }
@@ -10677,7 +10672,7 @@  build_ecmp_route_flow(struct lflow_table *lflows, struct ovn_datapath *od,
                       struct lflow_ref *lflow_ref)
 
 {
-    bool is_ipv4 = IN6_IS_ADDR_V4MAPPED(&eg->prefix);
+    bool is_ipv4_prefix = IN6_IS_ADDR_V4MAPPED(&eg->prefix);
     uint16_t priority;
     struct ecmp_route_list_node *er;
     struct ds route_match = DS_EMPTY_INITIALIZER;
@@ -10686,7 +10681,8 @@  build_ecmp_route_flow(struct lflow_table *lflows, struct ovn_datapath *od,
     int ofs = !strcmp(eg->origin, ROUTE_ORIGIN_CONNECTED) ?
         ROUTE_PRIO_OFFSET_CONNECTED: ROUTE_PRIO_OFFSET_STATIC;
     build_route_match(NULL, eg->route_table_id, prefix_s, eg->plen,
-                      eg->is_src_route, is_ipv4, &route_match, &priority, ofs);
+                      eg->is_src_route, is_ipv4_prefix, &route_match,
+                      &priority, ofs);
     free(prefix_s);
 
     struct ds actions = DS_EMPTY_INITIALIZER;
@@ -10719,7 +10715,8 @@  build_ecmp_route_flow(struct lflow_table *lflows, struct ovn_datapath *od,
         /* Find the outgoing port. */
         const char *lrp_addr_s = NULL;
         struct ovn_port *out_port = NULL;
-        if (!find_static_route_outport(od, lr_ports, route, is_ipv4,
+        if (!find_static_route_outport(od, lr_ports, route,
+                                       route_->is_ipv4_nexthop,
                                        &lrp_addr_s, &out_port)) {
             continue;
         }
@@ -10744,9 +10741,10 @@  build_ecmp_route_flow(struct lflow_table *lflows, struct ovn_datapath *od,
                       "eth.src = %s; "
                       "outport = %s; "
                       "next;",
-                      is_ipv4 ? REG_NEXT_HOP_IPV4 : REG_NEXT_HOP_IPV6,
+                      route_->is_ipv4_nexthop ?
+                          REG_NEXT_HOP_IPV4 : REG_NEXT_HOP_IPV6,
                       route->nexthop,
-                      is_ipv4 ? REG_SRC_IPV4 : REG_SRC_IPV6,
+                      route_->is_ipv4_nexthop ? REG_SRC_IPV4 : REG_SRC_IPV6,
                       lrp_addr_s,
                       out_port->lrp_networks.ea_s,
                       out_port->json_key);
@@ -10766,15 +10764,15 @@  add_route(struct lflow_table *lflows, struct ovn_datapath *od,
           const char *network_s, int plen, const char *gateway,
           bool is_src_route, const uint32_t rtb_id,
           const struct ovsdb_idl_row *stage_hint, bool is_discard_route,
-          int ofs, struct lflow_ref *lflow_ref)
+          int ofs, struct lflow_ref *lflow_ref,
+          bool is_ipv4_prefix, bool is_ipv4_nexthop)
 {
-    bool is_ipv4 = strchr(network_s, '.') ? true : false;
     struct ds match = DS_EMPTY_INITIALIZER;
     uint16_t priority;
     const struct ovn_port *op_inport = NULL;
 
     /* IPv6 link-local addresses must be scoped to the local router port. */
-    if (!is_ipv4) {
+    if (!is_ipv4_prefix) {
         struct in6_addr network;
         ovs_assert(ipv6_parse(network_s, &network));
         if (in6_is_lla(&network)) {
@@ -10782,7 +10780,7 @@  add_route(struct lflow_table *lflows, struct ovn_datapath *od,
         }
     }
     build_route_match(op_inport, rtb_id, network_s, plen, is_src_route,
-                      is_ipv4, &match, &priority, ofs);
+                      is_ipv4_prefix, &match, &priority, ofs);
 
     struct ds common_actions = DS_EMPTY_INITIALIZER;
     struct ds actions = DS_EMPTY_INITIALIZER;
@@ -10790,11 +10788,12 @@  add_route(struct lflow_table *lflows, struct ovn_datapath *od,
         ds_put_cstr(&actions, debug_drop_action());
     } else {
         ds_put_format(&common_actions, REG_ECMP_GROUP_ID" = 0; %s = ",
-                      is_ipv4 ? REG_NEXT_HOP_IPV4 : REG_NEXT_HOP_IPV6);
+                      is_ipv4_nexthop ? REG_NEXT_HOP_IPV4 : REG_NEXT_HOP_IPV6);
         if (gateway && gateway[0]) {
             ds_put_cstr(&common_actions, gateway);
         } else {
-            ds_put_format(&common_actions, "ip%s.dst", is_ipv4 ? "4" : "6");
+            ds_put_format(&common_actions, "ip%s.dst",
+                          is_ipv4_prefix ? "4" : "6");
         }
         ds_put_format(&common_actions, "; "
                       "%s = %s; "
@@ -10802,7 +10801,7 @@  add_route(struct lflow_table *lflows, struct ovn_datapath *od,
                       "outport = %s; "
                       "flags.loopback = 1; "
                       "next;",
-                      is_ipv4 ? REG_SRC_IPV4 : REG_SRC_IPV6,
+                      is_ipv4_nexthop ? REG_SRC_IPV4 : REG_SRC_IPV6,
                       lrp_addr_s,
                       op->lrp_networks.ea_s,
                       op->json_key);
@@ -10854,7 +10853,8 @@  build_static_route_flow(struct lflow_table *lflows, struct ovn_datapath *od,
     add_route(lflows, route_->is_discard_route ? od : out_port->od, out_port,
               lrp_addr_s, prefix_s, route_->plen, route->nexthop,
               route_->is_src_route, route_->route_table_id, &route->header_,
-              route_->is_discard_route, ofs, lflow_ref);
+              route_->is_discard_route, ofs, lflow_ref,
+              route_->is_ipv4_prefix, route_->is_ipv4_nexthop);
 
     free(prefix_s);
 }
@@ -12576,7 +12576,7 @@  build_ip_routing_flows_for_lrp(
                   op->lrp_networks.ipv4_addrs[i].network_s,
                   op->lrp_networks.ipv4_addrs[i].plen, NULL, false, 0,
                   &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED,
-                  lflow_ref);
+                  lflow_ref, true, true);
     }
 
     for (int i = 0; i < op->lrp_networks.n_ipv6_addrs; i++) {
@@ -12584,7 +12584,7 @@  build_ip_routing_flows_for_lrp(
                   op->lrp_networks.ipv6_addrs[i].network_s,
                   op->lrp_networks.ipv6_addrs[i].plen, NULL, false, 0,
                   &op->nbrp->header_, false, ROUTE_PRIO_OFFSET_CONNECTED,
-                  lflow_ref);
+                  lflow_ref, false, false);
     }
 }
 
@@ -15361,7 +15361,7 @@  build_routable_flows_for_router_port(
                               laddrs->ipv4_addrs[k].plen, NULL, false, 0,
                               &router_port->nbrp->header_, false,
                               ROUTE_PRIO_OFFSET_CONNECTED,
-                              lrp->stateful_lflow_ref);
+                              lrp->stateful_lflow_ref, true, true);
                 }
             }
         }
diff --git a/tests/ovn-nbctl.at b/tests/ovn-nbctl.at
index 5248e6c76..4a219ab61 100644
--- a/tests/ovn-nbctl.at
+++ b/tests/ovn-nbctl.at
@@ -1757,7 +1757,7 @@  AT_CHECK([ovn-nbctl lr-route-add lr0 10.0.0.1/24 11.0.0.2])
 AT_CHECK([ovn-nbctl lr-route-add lr0 10.0.10.0/24 lp0])
 AT_CHECK([ovn-nbctl --bfd lr-route-add lr0 10.0.20.0/24 11.0.2.1 lp0])
 AT_CHECK([ovn-nbctl lr-route-add lr0 10.0.10.0/24 lp1], [1], [],
-  [ovn-nbctl: bad IPv4 nexthop argument: lp1
+  [ovn-nbctl: bad nexthop argument: lp1
 ])
 
 dnl Add overlapping route with 10.0.0.1/24
@@ -1771,13 +1771,13 @@  AT_CHECK([ovn-nbctl lr-route-add lr0 10.0.0.111/24a 11.0.0.1], [1], [],
   [ovn-nbctl: bad prefix argument: 10.0.0.111/24a
 ])
 AT_CHECK([ovn-nbctl lr-route-add lr0 10.0.0.111/24 11.0.0.1a], [1], [],
-  [ovn-nbctl: bad IPv4 nexthop argument: 11.0.0.1a
+  [ovn-nbctl: bad nexthop argument: 11.0.0.1a
 ])
 AT_CHECK([ovn-nbctl lr-route-add lr0 10.0.0.111/24 11.0.0.1/24], [1], [],
-  [ovn-nbctl: bad IPv4 nexthop argument: 11.0.0.1/24
+  [ovn-nbctl: bad nexthop argument: 11.0.0.1/24
 ])
 AT_CHECK([ovn-nbctl lr-route-add lr0 2001:0db8:1::/64 2001:0db8:0:f103::1/64], [1], [],
-  [ovn-nbctl: bad IPv6 nexthop argument: 2001:0db8:0:f103::1/64
+  [ovn-nbctl: bad nexthop argument: 2001:0db8:0:f103::1/64
 ])
 AT_CHECK([ovn-nbctl --ecmp lr-route-add lr0 20.0.0.0/24 discard], [1], [],
   [ovn-nbctl: ecmp is not valid for discard routes.
@@ -2005,6 +2005,24 @@  check ovn-nbctl lr-route-del lr0
 AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
 ])
 
+dnl Check IPv4 over v6 and IPv6 over v4 routes
+AT_CHECK([ovn-nbctl lr-route-add lr0 10.0.0.1/24 2001:0db8:0:f103::10])
+AT_CHECK([ovn-nbctl lr-route-add lr0 2001:0db8:0::/64 11.0.1.10])
+
+AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
+IPv4 Routes
+Route Table <main>:
+              10.0.0.0/24       2001:db8:0:f103::10 dst-ip
+
+IPv6 Routes
+Route Table <main>:
+            2001:db8::/64                 11.0.1.10 dst-ip
+])
+
+check ovn-nbctl lr-route-del lr0
+AT_CHECK([ovn-nbctl lr-route-list lr0], [0], [dnl
+])
+
 dnl Check IPv4 routes in route table
 check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 0.0.0.0/0 192.168.0.1
 check ovn-nbctl --route-table=rtb-1 lr-route-add lr0 10.0.1.1/24 11.0.1.1 lp0
diff --git a/tests/ovn.at b/tests/ovn.at
index dc6aafd53..bf7d5ef4b 100644
--- a/tests/ovn.at
+++ b/tests/ovn.at
@@ -37772,3 +37772,514 @@  OVS_WAIT_FOR_OUTPUT([as hv1 ovs-ofctl dump-flows br-int table=0 |grep priority=1
 OVN_CLEANUP([hv1])
 AT_CLEANUP
 ])
+
+OVN_FOR_EACH_NORTHD([
+AT_SETUP([2 HVs, 2 LS, 1 lport/LS, 2 peer LRs, IPv4 over IPv6])
+AT_SKIP_IF([test $HAVE_SCAPY = no])
+ovn_start
+
+# Logical network:
+# Two LRs - R1 and R2 that are connected to each other as peers in 2001:db8::/64
+# network. R1 has a switchs ls1 (192.168.1.0/24) connected to it.
+# R2 has ls2 (172.16.1.0/24) connected to it.
+
+ls1_lp1_mac="f0:00:00:01:02:03"
+rp_ls1_mac="00:00:00:01:02:03"
+rp_ls2_mac="00:00:00:01:02:04"
+ls2_lp1_mac="f0:00:00:01:02:04"
+
+ls1_lp1_ip="192.168.1.2"
+ls2_lp1_ip="172.16.1.2"
+
+check ovn-nbctl lr-add R1
+check ovn-nbctl lr-add R2
+
+check ovn-nbctl ls-add ls1
+check ovn-nbctl ls-add ls2
+
+# Connect ls1 to R1
+check ovn-nbctl lrp-add R1 ls1 $rp_ls1_mac 192.168.1.1/24
+
+check ovn-nbctl lsp-add ls1 rp-ls1 -- set Logical_Switch_Port rp-ls1 type=router \
+  options:router-port=ls1 addresses=\"$rp_ls1_mac\"
+
+# Connect ls2 to R2
+check ovn-nbctl lrp-add R2 ls2 $rp_ls2_mac 172.16.1.1/24
+
+check ovn-nbctl lsp-add ls2 rp-ls2 -- set Logical_Switch_Port rp-ls2 type=router \
+  options:router-port=ls2 addresses=\"$rp_ls2_mac\"
+
+# Connect R1 to R2
+check ovn-nbctl lrp-add R1 R1_R2 00:00:00:02:03:04 2001:db8::1/64 peer=R2_R1
+check ovn-nbctl lrp-add R2 R2_R1 00:00:00:02:03:05 2001:db8::2/64 peer=R1_R2
+
+AT_CHECK([ovn-nbctl lr-route-add R1 "0.0.0.0/0" 2001:db8::2])
+AT_CHECK([ovn-nbctl lr-route-add R2 "0.0.0.0/0" 2001:db8::1])
+
+# Create logical port ls1-lp1 in ls1
+check ovn-nbctl lsp-add ls1 ls1-lp1 \
+-- lsp-set-addresses ls1-lp1 "$ls1_lp1_mac $ls1_lp1_ip"
+
+# Create logical port ls2-lp1 in ls2
+check ovn-nbctl lsp-add ls2 ls2-lp1 \
+-- lsp-set-addresses ls2-lp1 "$ls2_lp1_mac $ls2_lp1_ip"
+
+# Create two hypervisor and create OVS ports corresponding to logical ports.
+net_add n1
+
+sim_add hv1
+as hv1
+check ovs-vsctl add-br br-phys
+ovn_attach n1 br-phys 192.168.0.1
+check ovs-vsctl -- add-port br-int hv1-vif1 -- \
+    set interface hv1-vif1 external-ids:iface-id=ls1-lp1 \
+    options:tx_pcap=hv1/vif1-tx.pcap \
+    options:rxq_pcap=hv1/vif1-rx.pcap \
+    ofport-request=1
+
+sim_add hv2
+as hv2
+check ovs-vsctl add-br br-phys
+ovn_attach n1 br-phys 192.168.0.2
+check ovs-vsctl -- add-port br-int hv2-vif1 -- \
+    set interface hv2-vif1 external-ids:iface-id=ls2-lp1 \
+    options:tx_pcap=hv2/vif1-tx.pcap \
+    options:rxq_pcap=hv2/vif1-rx.pcap \
+    ofport-request=1
+
+
+# Pre-populate the hypervisors' ARP tables so that we don't lose any
+# packets for ARP resolution (native tunneling doesn't queue packets
+# for ARP resolution).
+OVN_POPULATE_ARP
+
+# Allow some time for ovn-northd and ovn-controller to catch up.
+wait_for_ports_up
+check ovn-nbctl --wait=hv sync
+
+# Packet to send.
+packet=$(fmt_pkt "Ether(dst='${rp_ls1_mac}', src='${ls1_lp1_mac}')/ \
+                        IP(src='${ls1_lp1_ip}', dst='${ls2_lp1_ip}', ttl=64)/ \
+                        UDP(sport=53, dport=4369)")
+check as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 "$packet"
+
+# Packet to Expect
+# The TTL should be decremented by 2.
+expected=$(fmt_pkt "Ether(dst='${ls2_lp1_mac}', src='${rp_ls2_mac}')/ \
+                        IP(src='${ls1_lp1_ip}', dst='${ls2_lp1_ip}', ttl=62)/ \
+                        UDP(sport=53, dport=4369)")
+echo ${expected} > expected
+OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
+
+AT_CHECK([ovn-sbctl dump-flows | grep lr_in_arp_resolve | \
+grep "reg0 == 172.16.1.2" | wc -l], [0], [1
+])
+
+# Disable the ls2-lp1 port.
+check ovn-nbctl --wait=hv set logical_switch_port ls2-lp1 enabled=false
+
+AT_CHECK([ovn-sbctl dump-flows | grep lr_in_arp_resolve | \
+grep "reg0 == 172.16.1.2" | wc -l], [0], [0
+])
+
+# Send the same packet again and it should not be delivered
+check as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 "$packet"
+
+# The 2nd packet sent shound not be received.
+OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
+
+OVN_CLEANUP([hv1],[hv2])
+
+AT_CLEANUP
+])
+
+OVN_FOR_EACH_NORTHD([
+AT_SETUP([2 HVs, 2 LS, 1 lport/LS, LRs connected via LS, IPv4 over IPv6])
+AT_SKIP_IF([test $HAVE_SCAPY = no])
+ovn_start
+
+# Logical network:
+# Two LRs - R1 and R2 that are connected to ls-transfer in 2001:db8::/64
+# network. R1 has a switchs ls1 (192.168.1.0/24) connected to it.
+# R2 has ls2 (172.16.1.0/24) connected to it.
+
+ls1_lp1_mac="f0:00:00:01:02:03"
+rp_ls1_mac="00:00:00:01:02:03"
+rp_ls2_mac="00:00:00:01:02:04"
+ls2_lp1_mac="f0:00:00:01:02:04"
+
+ls1_lp1_ip="192.168.1.2"
+ls2_lp1_ip="172.16.1.2"
+
+check ovn-nbctl lr-add R1
+check ovn-nbctl lr-add R2
+
+check ovn-nbctl ls-add ls1
+check ovn-nbctl ls-add ls2
+check ovn-nbctl ls-add ls-transfer
+
+# Connect ls1 to R1
+check ovn-nbctl lrp-add R1 ls1 $rp_ls1_mac 192.168.1.1/24
+
+check ovn-nbctl lsp-add ls1 rp-ls1 -- set Logical_Switch_Port rp-ls1 type=router \
+  options:router-port=ls1 addresses=\"$rp_ls1_mac\"
+
+# Connect ls2 to R2
+check ovn-nbctl lrp-add R2 ls2 $rp_ls2_mac 172.16.1.1/24
+
+check ovn-nbctl lsp-add ls2 rp-ls2 -- set Logical_Switch_Port rp-ls2 type=router \
+  options:router-port=ls2 addresses=\"$rp_ls2_mac\"
+
+# Connect R1 to R2
+check ovn-nbctl lrp-add R1 R1_ls-transfer 00:00:00:02:03:04 2001:db8::1/64
+check ovn-nbctl lrp-add R2 R2_ls-transfer 00:00:00:02:03:05 2001:db8::2/64
+
+check ovn-nbctl lsp-add ls-transfer ls-transfer_r1 -- \
+  set Logical_Switch_Port ls-transfer_r1 type=router \
+  options:router-port=R1_ls-transfer addresses=\"router\"
+check ovn-nbctl lsp-add ls-transfer ls-transfer_r2 -- \
+  set Logical_Switch_Port ls-transfer_r2 type=router \
+  options:router-port=R2_ls-transfer addresses=\"router\"
+
+AT_CHECK([ovn-nbctl lr-route-add R1 "0.0.0.0/0" 2001:db8::2])
+AT_CHECK([ovn-nbctl lr-route-add R2 "0.0.0.0/0" 2001:db8::1])
+
+# Create logical port ls1-lp1 in ls1
+check ovn-nbctl lsp-add ls1 ls1-lp1 \
+-- lsp-set-addresses ls1-lp1 "$ls1_lp1_mac $ls1_lp1_ip"
+
+# Create logical port ls2-lp1 in ls2
+check ovn-nbctl lsp-add ls2 ls2-lp1 \
+-- lsp-set-addresses ls2-lp1 "$ls2_lp1_mac $ls2_lp1_ip"
+
+# Create two hypervisor and create OVS ports corresponding to logical ports.
+net_add n1
+
+sim_add hv1
+as hv1
+check ovs-vsctl add-br br-phys
+ovn_attach n1 br-phys 192.168.0.1
+check ovs-vsctl -- add-port br-int hv1-vif1 -- \
+    set interface hv1-vif1 external-ids:iface-id=ls1-lp1 \
+    options:tx_pcap=hv1/vif1-tx.pcap \
+    options:rxq_pcap=hv1/vif1-rx.pcap \
+    ofport-request=1
+
+sim_add hv2
+as hv2
+check ovs-vsctl add-br br-phys
+ovn_attach n1 br-phys 192.168.0.2
+check ovs-vsctl -- add-port br-int hv2-vif1 -- \
+    set interface hv2-vif1 external-ids:iface-id=ls2-lp1 \
+    options:tx_pcap=hv2/vif1-tx.pcap \
+    options:rxq_pcap=hv2/vif1-rx.pcap \
+    ofport-request=1
+
+
+# Pre-populate the hypervisors' ARP tables so that we don't lose any
+# packets for ARP resolution (native tunneling doesn't queue packets
+# for ARP resolution).
+OVN_POPULATE_ARP
+
+# Allow some time for ovn-northd and ovn-controller to catch up.
+wait_for_ports_up
+check ovn-nbctl --wait=hv sync
+
+# Packet to send.
+packet=$(fmt_pkt "Ether(dst='${rp_ls1_mac}', src='${ls1_lp1_mac}')/ \
+                        IP(src='${ls1_lp1_ip}', dst='${ls2_lp1_ip}', ttl=64)/ \
+                        UDP(sport=53, dport=4369)")
+check as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 "$packet"
+
+# Packet to Expect
+# The TTL should be decremented by 2.
+expected=$(fmt_pkt "Ether(dst='${ls2_lp1_mac}', src='${rp_ls2_mac}')/ \
+                        IP(src='${ls1_lp1_ip}', dst='${ls2_lp1_ip}', ttl=62)/ \
+                        UDP(sport=53, dport=4369)")
+echo ${expected} > expected
+OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
+
+AT_CHECK([ovn-sbctl dump-flows | grep lr_in_arp_resolve | \
+grep "reg0 == 172.16.1.2" | wc -l], [0], [1
+])
+
+# Disable the ls2-lp1 port.
+check ovn-nbctl --wait=hv set logical_switch_port ls2-lp1 enabled=false
+
+AT_CHECK([ovn-sbctl dump-flows | grep lr_in_arp_resolve | \
+grep "reg0 == 172.16.1.2" | wc -l], [0], [0
+])
+
+# Send the same packet again and it should not be delivered
+check as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 "$packet"
+
+# The 2nd packet sent shound not be received.
+OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
+
+OVN_CLEANUP([hv1],[hv2])
+
+AT_CLEANUP
+])
+
+OVN_FOR_EACH_NORTHD([
+AT_SETUP([2 HVs, 2 LS, 1 lport/LS, LRs connected via LS, IPv4 over IPv6, ECMP])
+AT_SKIP_IF([test $HAVE_SCAPY = no])
+ovn_start
+
+# Logical network:
+# Two LRs - R1 and R2 that are connected to ls-transfer1 and lr-transfer2 in
+# 2001:db8:1::/64 and 2001:db8:2::/64
+# network. R1 has a switchs ls1 (192.168.1.0/24) connected to it.
+# R2 has ls2 (172.16.1.0/24) connected to it.
+
+ls1_lp1_mac="f0:00:00:01:02:03"
+rp_ls1_mac="00:00:00:01:02:03"
+rp_ls2_mac="00:00:00:01:02:04"
+ls2_lp1_mac="f0:00:00:01:02:04"
+
+ls1_lp1_ip="192.168.1.2"
+ls2_lp1_ip="172.16.1.2"
+
+check ovn-nbctl lr-add R1
+check ovn-nbctl lr-add R2
+
+check ovn-nbctl ls-add ls1
+check ovn-nbctl ls-add ls2
+check ovn-nbctl ls-add ls-transfer1
+check ovn-nbctl ls-add ls-transfer2
+
+# Connect ls1 to R1
+check ovn-nbctl lrp-add R1 ls1 $rp_ls1_mac 192.168.1.1/24
+
+check ovn-nbctl lsp-add ls1 rp-ls1 -- set Logical_Switch_Port rp-ls1 type=router \
+  options:router-port=ls1 addresses=\"$rp_ls1_mac\"
+
+# Connect ls2 to R2
+check ovn-nbctl lrp-add R2 ls2 $rp_ls2_mac 172.16.1.1/24
+
+check ovn-nbctl lsp-add ls2 rp-ls2 -- set Logical_Switch_Port rp-ls2 type=router \
+  options:router-port=ls2 addresses=\"$rp_ls2_mac\"
+
+# Connect R1 to R2 (ls-transfer1)
+check ovn-nbctl lrp-add R1 R1_ls-transfer1 00:00:00:02:03:04 2001:db8:1::1/64
+check ovn-nbctl lrp-add R2 R2_ls-transfer1 00:00:00:02:03:05 2001:db8:1::2/64
+
+check ovn-nbctl lsp-add ls-transfer1 ls-transfer1_r1 -- \
+  set Logical_Switch_Port ls-transfer1_r1 type=router \
+  options:router-port=R1_ls-transfer1 addresses=\"router\"
+check ovn-nbctl lsp-add ls-transfer1 ls-transfer1_r2 -- \
+  set Logical_Switch_Port ls-transfer1_r2 type=router \
+  options:router-port=R2_ls-transfer1 addresses=\"router\"
+
+# Connect R1 to R2 (ls-transfer2)
+check ovn-nbctl lrp-add R1 R1_ls-transfer2 00:00:00:02:03:14 2001:db8:2::1/64
+check ovn-nbctl lrp-add R2 R2_ls-transfer2 00:00:00:02:03:15 2001:db8:2::2/64
+
+check ovn-nbctl lsp-add ls-transfer2 ls-transfer2_r1 -- \
+  set Logical_Switch_Port ls-transfer2_r1 type=router \
+  options:router-port=R1_ls-transfer2 addresses=\"router\"
+check ovn-nbctl lsp-add ls-transfer2 ls-transfer2_r2 -- \
+  set Logical_Switch_Port ls-transfer2_r2 type=router \
+  options:router-port=R2_ls-transfer2 addresses=\"router\"
+
+AT_CHECK([ovn-nbctl lr-route-add R1 "0.0.0.0/0" 2001:db8:1::2])
+AT_CHECK([ovn-nbctl --ecmp lr-route-add R1 "0.0.0.0/0" 2001:db8:2::2])
+AT_CHECK([ovn-nbctl lr-route-add R2 "0.0.0.0/0" 2001:db8:1::1])
+AT_CHECK([ovn-nbctl --ecmp lr-route-add R2 "0.0.0.0/0" 2001:db8:2::1])
+
+# Create logical port ls1-lp1 in ls1
+check ovn-nbctl lsp-add ls1 ls1-lp1 \
+-- lsp-set-addresses ls1-lp1 "$ls1_lp1_mac $ls1_lp1_ip"
+
+# Create logical port ls2-lp1 in ls2
+check ovn-nbctl lsp-add ls2 ls2-lp1 \
+-- lsp-set-addresses ls2-lp1 "$ls2_lp1_mac $ls2_lp1_ip"
+
+# Create two hypervisor and create OVS ports corresponding to logical ports.
+net_add n1
+
+sim_add hv1
+as hv1
+check ovs-vsctl add-br br-phys
+ovn_attach n1 br-phys 192.168.0.1
+check ovs-vsctl -- add-port br-int hv1-vif1 -- \
+    set interface hv1-vif1 external-ids:iface-id=ls1-lp1 \
+    options:tx_pcap=hv1/vif1-tx.pcap \
+    options:rxq_pcap=hv1/vif1-rx.pcap \
+    ofport-request=1
+
+sim_add hv2
+as hv2
+check ovs-vsctl add-br br-phys
+ovn_attach n1 br-phys 192.168.0.2
+check ovs-vsctl -- add-port br-int hv2-vif1 -- \
+    set interface hv2-vif1 external-ids:iface-id=ls2-lp1 \
+    options:tx_pcap=hv2/vif1-tx.pcap \
+    options:rxq_pcap=hv2/vif1-rx.pcap \
+    ofport-request=1
+
+
+# Pre-populate the hypervisors' ARP tables so that we don't lose any
+# packets for ARP resolution (native tunneling doesn't queue packets
+# for ARP resolution).
+OVN_POPULATE_ARP
+
+# Allow some time for ovn-northd and ovn-controller to catch up.
+wait_for_ports_up
+check ovn-nbctl --wait=hv sync
+
+# Packet to send.
+packet=$(fmt_pkt "Ether(dst='${rp_ls1_mac}', src='${ls1_lp1_mac}')/ \
+                        IP(src='${ls1_lp1_ip}', dst='${ls2_lp1_ip}', ttl=64)/ \
+                        UDP(sport=53, dport=4369)")
+check as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 "$packet"
+
+# Packet to Expect
+# The TTL should be decremented by 2.
+expected=$(fmt_pkt "Ether(dst='${ls2_lp1_mac}', src='${rp_ls2_mac}')/ \
+                        IP(src='${ls1_lp1_ip}', dst='${ls2_lp1_ip}', ttl=62)/ \
+                        UDP(sport=53, dport=4369)")
+echo ${expected} > expected
+OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
+
+AT_CHECK([ovn-sbctl dump-flows | grep lr_in_arp_resolve | \
+grep "reg0 == 172.16.1.2" | wc -l], [0], [1
+])
+
+# Disable the ls2-lp1 port.
+check ovn-nbctl --wait=hv set logical_switch_port ls2-lp1 enabled=false
+
+AT_CHECK([ovn-sbctl dump-flows | grep lr_in_arp_resolve | \
+grep "reg0 == 172.16.1.2" | wc -l], [0], [0
+])
+
+# Send the same packet again and it should not be delivered
+check as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 "$packet"
+
+# The 2nd packet sent shound not be received.
+OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
+
+OVN_CLEANUP([hv1],[hv2])
+
+AT_CLEANUP
+])
+
+OVN_FOR_EACH_NORTHD([
+AT_SETUP([2 HVs, 2 LS, 1 lport/LS, 2 peer LRs, IPv6 over IPv4])
+AT_SKIP_IF([test $HAVE_SCAPY = no])
+ovn_start
+
+# Logical network:
+# Two LRs - R1 and R2 that are connected to each other as peers in 10.0.0.0/24
+# network. R1 has a switchs ls1 (2001:db8:1::/64) connected to it.
+# R2 has ls2 (2001:db8:2::/64) connected to it.
+
+ls1_lp1_mac="f0:00:00:01:02:03"
+rp_ls1_mac="00:00:00:01:02:03"
+rp_ls2_mac="00:00:00:01:02:04"
+ls2_lp1_mac="f0:00:00:01:02:04"
+
+ls1_lp1_ip="2001:db8:1::2"
+ls2_lp1_ip="2001:db8:2::2"
+
+check ovn-nbctl lr-add R1
+check ovn-nbctl lr-add R2
+
+check ovn-nbctl ls-add ls1
+check ovn-nbctl ls-add ls2
+
+# Connect ls1 to R1
+check ovn-nbctl lrp-add R1 ls1 $rp_ls1_mac 2001:db8:1::1/64
+
+check ovn-nbctl lsp-add ls1 rp-ls1 -- set Logical_Switch_Port rp-ls1 type=router \
+  options:router-port=ls1 addresses=\"$rp_ls1_mac\"
+
+# Connect ls2 to R2
+check ovn-nbctl lrp-add R2 ls2 $rp_ls2_mac 2001:db8:2::1/64
+
+check ovn-nbctl lsp-add ls2 rp-ls2 -- set Logical_Switch_Port rp-ls2 type=router \
+  options:router-port=ls2 addresses=\"$rp_ls2_mac\"
+
+# Connect R1 to R2
+check ovn-nbctl lrp-add R1 R1_R2 00:00:00:02:03:04 10.0.0.1/24 peer=R2_R1
+check ovn-nbctl lrp-add R2 R2_R1 00:00:00:02:03:05 10.0.0.2/24 peer=R1_R2
+
+AT_CHECK([ovn-nbctl lr-route-add R1 "::/0" 10.0.0.2])
+AT_CHECK([ovn-nbctl lr-route-add R2 "::/0" 10.0.0.1])
+
+# Create logical port ls1-lp1 in ls1
+check ovn-nbctl lsp-add ls1 ls1-lp1 \
+-- lsp-set-addresses ls1-lp1 "$ls1_lp1_mac $ls1_lp1_ip"
+
+# Create logical port ls2-lp1 in ls2
+check ovn-nbctl lsp-add ls2 ls2-lp1 \
+-- lsp-set-addresses ls2-lp1 "$ls2_lp1_mac $ls2_lp1_ip"
+
+# Create two hypervisor and create OVS ports corresponding to logical ports.
+net_add n1
+
+sim_add hv1
+as hv1
+check ovs-vsctl add-br br-phys
+ovn_attach n1 br-phys 192.168.0.1
+check ovs-vsctl -- add-port br-int hv1-vif1 -- \
+    set interface hv1-vif1 external-ids:iface-id=ls1-lp1 \
+    options:tx_pcap=hv1/vif1-tx.pcap \
+    options:rxq_pcap=hv1/vif1-rx.pcap \
+    ofport-request=1
+
+sim_add hv2
+as hv2
+check ovs-vsctl add-br br-phys
+ovn_attach n1 br-phys 192.168.0.2
+check ovs-vsctl -- add-port br-int hv2-vif1 -- \
+    set interface hv2-vif1 external-ids:iface-id=ls2-lp1 \
+    options:tx_pcap=hv2/vif1-tx.pcap \
+    options:rxq_pcap=hv2/vif1-rx.pcap \
+    ofport-request=1
+
+
+# Pre-populate the hypervisors' ARP tables so that we don't lose any
+# packets for ARP resolution (native tunneling doesn't queue packets
+# for ARP resolution).
+OVN_POPULATE_ARP
+
+# Allow some time for ovn-northd and ovn-controller to catch up.
+wait_for_ports_up
+check ovn-nbctl --wait=hv sync
+
+# Packet to send.
+packet=$(fmt_pkt "Ether(dst='${rp_ls1_mac}', src='${ls1_lp1_mac}')/ \
+                        IPv6(src='${ls1_lp1_ip}', dst='${ls2_lp1_ip}', hlim=64)/ \
+                        UDP(sport=53, dport=4369)")
+check as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 "$packet"
+
+# Packet to Expect
+# The TTL should be decremented by 2.
+expected=$(fmt_pkt "Ether(dst='${ls2_lp1_mac}', src='${rp_ls2_mac}')/ \
+                        IPv6(src='${ls1_lp1_ip}', dst='${ls2_lp1_ip}', hlim=62)/ \
+                        UDP(sport=53, dport=4369)")
+echo ${expected} > expected
+OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
+
+AT_CHECK([ovn-sbctl dump-flows | grep lr_in_arp_resolve | \
+grep "xxreg0 == 2001:db8:2::2" | wc -l], [0], [1
+])
+
+# Disable the ls2-lp1 port.
+check ovn-nbctl --wait=hv set logical_switch_port ls2-lp1 enabled=false
+
+AT_CHECK([ovn-sbctl dump-flows | grep lr_in_arp_resolve | \
+grep "xxreg0 == 2001:db8:2::2" | wc -l], [0], [0
+])
+
+# Send the same packet again and it should not be delivered
+check as hv1 ovs-appctl netdev-dummy/receive hv1-vif1 "$packet"
+
+# The 2nd packet sent shound not be received.
+OVN_CHECK_PACKETS([hv2/vif1-tx.pcap], [expected])
+
+OVN_CLEANUP([hv1],[hv2])
+
+AT_CLEANUP
+])
diff --git a/utilities/ovn-nbctl.c b/utilities/ovn-nbctl.c
index 25eb86f7f..f827b2ad9 100644
--- a/utilities/ovn-nbctl.c
+++ b/utilities/ovn-nbctl.c
@@ -4546,11 +4546,9 @@  nbctl_lr_route_add(struct ctl_context *ctx)
     }
 
     char *route_table = shash_find_data(&ctx->options, "--route-table");
-    bool v6_prefix = false;
     prefix = normalize_ipv4_prefix_str(ctx->argv[2]);
     if (!prefix) {
         prefix = normalize_ipv6_prefix_str(ctx->argv[2]);
-        v6_prefix = true;
     }
     if (!prefix) {
         ctl_error(ctx, "bad prefix argument: %s", ctx->argv[2]);
@@ -4561,15 +4559,15 @@  nbctl_lr_route_add(struct ctl_context *ctx)
     if (is_discard_route) {
         next_hop = xasprintf("discard");
     } else {
-        next_hop = v6_prefix
-            ? normalize_ipv6_addr_str(ctx->argv[3])
-            : normalize_ipv4_addr_str(ctx->argv[3]);
+        next_hop = normalize_ipv4_addr_str(ctx->argv[3]);
+        if (!next_hop) {
+            next_hop = normalize_ipv6_addr_str(ctx->argv[3]);
+        }
         if (!next_hop) {
             /* check if it is a output port. */
             error = lrp_by_name_or_uuid(ctx, ctx->argv[3], true, &out_lrp);
             if (error) {
-                ctl_error(ctx, "bad %s nexthop argument: %s",
-                          v6_prefix ? "IPv6" : "IPv4", ctx->argv[3]);
+                ctl_error(ctx, "bad nexthop argument: %s", ctx->argv[3]);
                 free(error);
                 goto cleanup;
             }