From patchwork Mon Feb 27 13:12:57 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guoshuai Li X-Patchwork-Id: 732815 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3vX2GF577Zz9s81 for ; Tue, 28 Feb 2017 00:13:41 +1100 (AEDT) Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id 46CCCB44; Mon, 27 Feb 2017 13:13:37 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@mail.linuxfoundation.org Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 87667415 for ; Mon, 27 Feb 2017 13:13:33 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from smtp2203-239.mail.aliyun.com (smtp2203-239.mail.aliyun.com [121.197.203.239]) by smtp1.linuxfoundation.org (Postfix) with ESMTP id 8A1C819F for ; Mon, 27 Feb 2017 13:13:18 +0000 (UTC) X-Alimail-AntiSpam: AC=CONTINUE; BC=0.07445402|-1; FP=0|0|0|0|0|-1|-1|-1; HT=e02c03291; MF=ligs@dtdream.com; NM=1; PH=DS; RN=2; RT=2; SC=91.7; SR=0; TI=SMTPD_---.7iY.sXh_1488201187; Received: from localhost.localdomain(mailfrom:ligs@dtdream.com ip:111.198.29.132) by smtp.aliyun-inc.com(10.147.42.253); Mon, 27 Feb 2017 21:13:07 +0800 From: Guoshuai Li To: ovs-dev@openvswitch.org Date: Mon, 27 Feb 2017 21:12:57 +0800 Message-Id: <20170227131257.4852-1-ligs@dtdream.com> X-Mailer: git-send-email 2.10.1.windows.1 X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: [ovs-dev] [PATCH] Support multiple logical routing port configuration "redirect-chassis" on a distributed router X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: ovs-dev-bounces@openvswitch.org Errors-To: ovs-dev-bounces@openvswitch.org The main application scenario of this patch is that the user flow wants to different destination addresses through different external networks. This scenario requires a distributed route to be associated with multiple external network logical switches. In a distributed router, the NAT logical flow table is generated based on the external IP lookup distributed router port, otherwise not generated. When the destination address of the packet is an external IP of the NAT rule, and the ingress port is not a gateway, it is necessary to route the actual outgoing port. Signed-off-by: Guoshuai Li Co-authored-by: Dong Jun --- ovn/northd/ovn-northd.8.xml | 22 ++--- ovn/northd/ovn-northd.c | 232 +++++++++++++++++++++++--------------------- ovn/ovn-nb.xml | 12 ++- 3 files changed, 137 insertions(+), 129 deletions(-) diff --git a/ovn/northd/ovn-northd.8.xml b/ovn/northd/ovn-northd.8.xml index ab8fd88..d9056ef 100644 --- a/ovn/northd/ovn-northd.8.xml +++ b/ovn/northd/ovn-northd.8.xml @@ -1549,16 +1549,6 @@ icmp4 {

  • - For distributed logical routers where one of the logical router - ports specifies a redirect-chassis, a priority-300 - logical flow with match REGBIT_NAT_REDIRECT == 1 has - actions ip.ttl--; next;. The outport - will be set later in the Gateway Redirect table. -

    -
    • - -
  • -

    IPv4 routing table. For each route to IPv4 network N with netmask M, on router port P with IP address A and Ethernet @@ -1644,12 +1634,12 @@ next;

    • - For distributed logical routers where one of the logical router - ports specifies a redirect-chassis, a priority-200 - logical flow with match REGBIT_NAT_REDIRECT == 1 has - actions eth.dst = E; next;, where - E is the ethernet address of the router's distributed - gateway port. + For distributed logical routers where router port P + specifies a redirect-chassis, a priority-200 + logical flow with match REGBIT_NAT_REDIRECT == 1 + and outport == P has actions + eth.dst = E; next;, where E + is the ethernet address of the router's distributed gateway port.

      • diff --git a/ovn/northd/ovn-northd.c b/ovn/northd/ovn-northd.c index 03dc850..8477e23 100644 --- a/ovn/northd/ovn-northd.c +++ b/ovn/northd/ovn-northd.c @@ -398,12 +398,10 @@ struct ovn_datapath { /* OVN northd only needs to know about the logical router gateway port for * NAT on a distributed router. This "distributed gateway port" is - * populated only when there is a "redirect-chassis" specified for one of - * the ports on the logical router. Otherwise this will be NULL. */ - struct ovn_port *l3dgw_port; - /* The "derived" OVN port representing the instance of l3dgw_port on - * the "redirect-chassis". */ - struct ovn_port *l3redirect_port; + * populated only when there is a "redirect-chassis" specified for the + * ports on the logical router. Otherwise this will be NULL. */ + struct ovn_port **l3dgw_ports; + size_t n_l3dgw_ports; }; struct macam_node { @@ -683,6 +681,10 @@ struct ovn_port { bool derived; /* Indicates whether this is an additional port * derived from nbsp or nbrp. */ + /* The "derived" OVN port representing the instance of l3dgw_port on + * the "redirect-chassis". Otherwise this will be NULL. */ + struct ovn_port *l3redirect_port; + /* The port's peer: * * - A switch port S of type "router" has a router port R as a peer, @@ -1402,16 +1404,11 @@ join_logical_ports(struct northd_context *ctx, /* Set l3dgw_port and l3redirect_port in od, for later * use during flow creation. */ - if (od->l3dgw_port || od->l3redirect_port) { - static struct vlog_rate_limit rl - = VLOG_RATE_LIMIT_INIT(1, 1); - VLOG_WARN_RL(&rl, "Bad configuration: multiple ports " - "with redirect-chassis on same logical " - "router %s", od->nbr->name); - continue; - } else { - od->l3dgw_port = op; - od->l3redirect_port = crp; + if (!op->l3redirect_port) { + op->l3redirect_port = crp; + od->l3dgw_ports = xrealloc(od->l3dgw_ports, + sizeof *od->l3dgw_ports * (od->n_l3dgw_ports + 1)); + od->l3dgw_ports[od->n_l3dgw_ports++] = op; } } } @@ -3300,14 +3297,12 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports, ds_clear(&match); ds_put_format(&match, "eth.dst == "ETH_ADDR_FMT, ETH_ADDR_ARGS(mac)); - if (op->peer->od->l3dgw_port - && op->peer == op->peer->od->l3dgw_port - && op->peer->od->l3redirect_port) { + if (op->peer->l3redirect_port) { /* The destination lookup flow for the router's * distributed gateway port MAC address should only be * programmed on the "redirect-chassis". */ ds_put_format(&match, " && is_chassis_resident(%s)", - op->peer->od->l3redirect_port->json_key); + op->peer->l3redirect_port->json_key); } ds_clear(&actions); @@ -3317,8 +3312,7 @@ build_lswitch_flows(struct hmap *datapaths, struct hmap *ports, /* Add ethernet addresses specified in NAT rules on * distributed logical routers. */ - if (op->peer->od->l3dgw_port - && op->peer == op->peer->od->l3dgw_port) { + if (op->peer->l3redirect_port) { for (int i = 0; i < op->peer->od->nbr->n_nat; i++) { const struct nbrec_nat *nat = op->peer->od->nbr->nat[i]; @@ -3473,6 +3467,22 @@ find_lrp_member_ip(const struct ovn_port *op, const char *ip_s) return NULL; } +static struct ovn_port * +find_l3dgw_port(struct ovn_datapath *od, const char *external_ip) +{ + for (size_t i = 0; i < od->n_l3dgw_ports; i++) { + struct ovn_port *l3dgw_port = od->l3dgw_ports[i]; + if (find_lrp_member_ip(l3dgw_port, external_ip)){ + return l3dgw_port; + } + } + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "can not find l3dgw port with redirect-chassis " + "for nat, external ip %s in router "UUID_FMT"", + external_ip, UUID_ARGS(&od->key)); + return NULL; +} + static void add_route(struct hmap *lflows, const struct ovn_port *op, const char *lrp_addr_s, const char *network_s, int plen, @@ -3788,12 +3798,11 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap *ports, ds_clear(&match); ds_put_format(&match, "eth.dst == %s && inport == %s", op->lrp_networks.ea_s, op->json_key); - if (op->od->l3dgw_port && op == op->od->l3dgw_port - && op->od->l3redirect_port) { + if (op->l3redirect_port) { /* Traffic with eth.dst = l3dgw_port->lrp_networks.ea_s * should only be received on the "redirect-chassis". */ ds_put_format(&match, " && is_chassis_resident(%s)", - op->od->l3redirect_port->json_key); + op->l3redirect_port->json_key); } ovn_lflow_add(lflows, op->od, S_ROUTER_IN_ADMISSION, 50, ds_cstr(&match), "next;"); @@ -3902,15 +3911,14 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap *ports, ds_put_format(&match, "inport == %s && arp.tpa == %s && arp.op == 1", op->json_key, op->lrp_networks.ipv4_addrs[i].addr_s); - if (op->od->l3dgw_port && op == op->od->l3dgw_port - && op->od->l3redirect_port) { + if (op->l3redirect_port) { /* Traffic with eth.src = l3dgw_port->lrp_networks.ea_s * should only be sent from the "redirect-chassis", so that * upstream MAC learning points to the "redirect-chassis". * Also need to avoid generation of multiple ARP responses * from different chassis. */ ds_put_format(&match, " && is_chassis_resident(%s)", - op->od->l3redirect_port->json_key); + op->l3redirect_port->json_key); } ds_clear(&actions); @@ -4046,7 +4054,7 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap *ports, "arp.op = 2; /* ARP reply */ " "arp.tha = arp.sha; "); - if (op->od->l3dgw_port && op == op->od->l3dgw_port) { + if (op->l3redirect_port) { struct eth_addr mac; if (nat->external_mac && eth_addr_from_string(nat->external_mac, &mac) @@ -4075,10 +4083,8 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap *ports, * upstream MAC learning points to the "redirect-chassis". * Also need to avoid generation of multiple ARP responses * from different chassis. */ - if (op->od->l3redirect_port) { - ds_put_format(&match, " && is_chassis_resident(%s)", - op->od->l3redirect_port->json_key); - } + ds_put_format(&match, " && is_chassis_resident(%s)", + op->l3redirect_port->json_key); } } else { ds_put_format(&actions, @@ -4188,15 +4194,14 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap *ports, op->lrp_networks.ipv6_addrs[i].addr_s, op->lrp_networks.ipv6_addrs[i].sn_addr_s, op->lrp_networks.ipv6_addrs[i].addr_s); - if (op->od->l3dgw_port && op == op->od->l3dgw_port - && op->od->l3redirect_port) { + if (op->l3redirect_port) { /* Traffic with eth.src = l3dgw_port->lrp_networks.ea_s * should only be sent from the "redirect-chassis", so that * upstream MAC learning points to the "redirect-chassis". * Also need to avoid generation of multiple ND replies * from different chassis. */ ds_put_format(&match, " && is_chassis_resident(%s)", - op->od->l3redirect_port->json_key); + op->l3redirect_port->json_key); } ds_clear(&actions); @@ -4237,7 +4242,7 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap *ports, /* NAT rules are only valid on Gateway routers and routers with * l3dgw_port (router has a port with "redirect-chassis" * specified). */ - if (!smap_get(&od->nbr->options, "chassis") && !od->l3dgw_port) { + if (!smap_get(&od->nbr->options, "chassis") && !od->n_l3dgw_ports) { continue; } @@ -4291,7 +4296,7 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap *ports, * satisfies the conditions for distributed NAT processing. */ bool distributed = false; struct eth_addr mac; - if (od->l3dgw_port && !strcmp(nat->type, "dnat_and_snat") && + if (od->n_l3dgw_ports && !strcmp(nat->type, "dnat_and_snat") && nat->logical_port && nat->external_mac) { if (eth_addr_from_string(nat->external_mac, &mac)) { distributed = true; @@ -4304,6 +4309,9 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap *ports, } } + /* find l3dgw port by external ip */ + struct ovn_port *l3dgw_port = find_l3dgw_port(od, nat->external_ip); + /* Ingress UNSNAT table: It is for already established connections' * reverse traffic. i.e., SNAT has already been done in egress * pipeline and now the packet has entered the ingress pipeline as @@ -4315,14 +4323,14 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap *ports, * egress pipeline. */ if (!strcmp(nat->type, "snat") || !strcmp(nat->type, "dnat_and_snat")) { - if (!od->l3dgw_port) { + if (!od->n_l3dgw_ports) { /* Gateway router. */ ds_clear(&match); ds_put_format(&match, "ip && ip4.dst == %s", nat->external_ip); ovn_lflow_add(lflows, od, S_ROUTER_IN_UNSNAT, 90, ds_cstr(&match), "ct_snat; next;"); - } else { + } else if (l3dgw_port) { /* Distributed router. */ /* Traffic received on l3dgw_port is subject to NAT. */ @@ -4330,12 +4338,12 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap *ports, ds_put_format(&match, "ip && ip4.dst == %s" " && inport == %s", nat->external_ip, - od->l3dgw_port->json_key); - if (!distributed && od->l3redirect_port) { + l3dgw_port->json_key); + if (!distributed && l3dgw_port->l3redirect_port) { /* Flows for NAT rules that are centralized are only * programmed on the "redirect-chassis". */ ds_put_format(&match, " && is_chassis_resident(%s)", - od->l3redirect_port->json_key); + l3dgw_port->l3redirect_port->json_key); } ovn_lflow_add(lflows, od, S_ROUTER_IN_UNSNAT, 100, ds_cstr(&match), "ct_snat;"); @@ -4357,7 +4365,7 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap *ports, * to a logical IP address. */ if (!strcmp(nat->type, "dnat") || !strcmp(nat->type, "dnat_and_snat")) { - if (!od->l3dgw_port) { + if (!od->n_l3dgw_ports) { /* Gateway router. */ /* Packet when it goes from the initiator to destination. * We need to set flags.loopback because the router can @@ -4377,7 +4385,7 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap *ports, nat->logical_ip); ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 100, ds_cstr(&match), ds_cstr(&actions)); - } else { + } else if (l3dgw_port) { /* Distributed router. */ /* Traffic received on l3dgw_port is subject to NAT. */ @@ -4385,12 +4393,12 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap *ports, ds_put_format(&match, "ip && ip4.dst == %s" " && inport == %s", nat->external_ip, - od->l3dgw_port->json_key); - if (!distributed && od->l3redirect_port) { + l3dgw_port->json_key); + if (!distributed && l3dgw_port->l3redirect_port) { /* Flows for NAT rules that are centralized are only * programmed on the "redirect-chassis". */ ds_put_format(&match, " && is_chassis_resident(%s)", - od->l3redirect_port->json_key); + l3dgw_port->l3redirect_port->json_key); } ds_clear(&actions); ds_put_format(&actions, "ct_dnat(%s);", @@ -4418,18 +4426,18 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap *ports, * Note that this only applies for NAT on a distributed router. * Undo DNAT on a gateway router is done in the ingress DNAT * pipeline stage. */ - if (od->l3dgw_port && (!strcmp(nat->type, "dnat") + if (l3dgw_port && (!strcmp(nat->type, "dnat") || !strcmp(nat->type, "dnat_and_snat"))) { ds_clear(&match); ds_put_format(&match, "ip && ip4.src == %s" " && outport == %s", nat->logical_ip, - od->l3dgw_port->json_key); - if (!distributed && od->l3redirect_port) { + l3dgw_port->json_key); + if (!distributed && l3dgw_port->l3redirect_port) { /* Flows for NAT rules that are centralized are only * programmed on the "redirect-chassis". */ ds_put_format(&match, " && is_chassis_resident(%s)", - od->l3redirect_port->json_key); + l3dgw_port->l3redirect_port->json_key); } ds_clear(&actions); if (distributed) { @@ -4446,7 +4454,7 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap *ports, * address. */ if (!strcmp(nat->type, "snat") || !strcmp(nat->type, "dnat_and_snat")) { - if (!od->l3dgw_port) { + if (!od->n_l3dgw_ports) { /* Gateway router. */ ds_clear(&match); ds_put_format(&match, "ip && ip4.src == %s", @@ -4460,18 +4468,18 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap *ports, ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, count_1bits(ntohl(mask)) + 1, ds_cstr(&match), ds_cstr(&actions)); - } else { + } else if (l3dgw_port) { /* Distributed router. */ ds_clear(&match); ds_put_format(&match, "ip && ip4.src == %s" " && outport == %s", nat->logical_ip, - od->l3dgw_port->json_key); - if (!distributed && od->l3redirect_port) { + l3dgw_port->json_key); + if (!distributed && l3dgw_port->l3redirect_port) { /* Flows for NAT rules that are centralized are only * programmed on the "redirect-chassis". */ ds_put_format(&match, " && is_chassis_resident(%s)", - od->l3redirect_port->json_key); + l3dgw_port->l3redirect_port->json_key); } ds_clear(&actions); if (distributed) { @@ -4495,15 +4503,18 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap *ports, * on the l3dgw_port instance where nat->logical_port is * resident. */ if (distributed) { - ds_clear(&match); - ds_put_format(&match, - "eth.dst == "ETH_ADDR_FMT" && inport == %s" - " && is_chassis_resident(\"%s\")", - ETH_ADDR_ARGS(mac), - od->l3dgw_port->json_key, - nat->logical_port); - ovn_lflow_add(lflows, od, S_ROUTER_IN_ADMISSION, 50, - ds_cstr(&match), "next;"); + for (size_t i = 0; i < od->n_l3dgw_ports; i++) { + struct ovn_port *l3dgw_port = od->l3dgw_ports[i]; + ds_clear(&match); + ds_put_format(&match, + "eth.dst == "ETH_ADDR_FMT" && inport == %s" + " && is_chassis_resident(\"%s\")", + ETH_ADDR_ARGS(mac), + l3dgw_port->json_key, + nat->logical_port); + ovn_lflow_add(lflows, od, S_ROUTER_IN_ADMISSION, 50, + ds_cstr(&match), "next;"); + } } /* Ingress Gateway Redirect Table: For NAT on a distributed @@ -4511,12 +4522,15 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap *ports, * flows indicate the presence of an applicable NAT rule that * can be applied in a distributed manner. */ if (distributed) { - ds_clear(&match); - ds_put_format(&match, "ip4.src == %s && outport == %s", - nat->logical_ip, - od->l3dgw_port->json_key); - ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 100, - ds_cstr(&match), "next;"); + for (size_t i = 0; i < od->n_l3dgw_ports; i++) { + struct ovn_port *l3dgw_port = od->l3dgw_ports[i]; + ds_clear(&match); + ds_put_format(&match, "ip4.src == %s && outport == %s", + nat->logical_ip, + l3dgw_port->json_key); + ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 100, + ds_cstr(&match), "next;"); + } } /* Egress Loopback table: For NAT on a distributed router. @@ -4524,12 +4538,12 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap *ports, * gateway port have ip.dst matching a NAT external IP, then * loop a clone of the packet back to the beginning of the * ingress pipeline with inport = outport. */ - if (od->l3dgw_port) { + if (l3dgw_port) { /* Distributed router. */ ds_clear(&match); ds_put_format(&match, "ip4.dst == %s && outport == %s", nat->external_ip, - od->l3dgw_port->json_key); + l3dgw_port->json_key); ds_clear(&actions); ds_put_format(&actions, "clone { ct_clear; " @@ -4546,7 +4560,7 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap *ports, } /* Handle force SNAT options set in the gateway router. */ - if (dnat_force_snat_ip && !od->l3dgw_port) { + if (dnat_force_snat_ip && !od->n_l3dgw_ports) { /* If a packet with destination IP address as that of the * gateway router (as set in options:dnat_force_snat_ip) is seen, * UNSNAT it. */ @@ -4565,7 +4579,7 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap *ports, ovn_lflow_add(lflows, od, S_ROUTER_OUT_SNAT, 100, ds_cstr(&match), ds_cstr(&actions)); } - if (lb_force_snat_ip && !od->l3dgw_port) { + if (lb_force_snat_ip && !od->n_l3dgw_ports) { /* If a packet with destination IP address as that of the * gateway router (as set in options:lb_force_snat_ip) is seen, * UNSNAT it. */ @@ -4584,7 +4598,7 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap *ports, ds_cstr(&match), ds_cstr(&actions)); } - if (!od->l3dgw_port) { + if (!od->n_l3dgw_ports) { /* For gateway router, re-circulate every packet through * the DNAT zone. This helps with two things. * @@ -4602,36 +4616,33 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap *ports, ovn_lflow_add(lflows, od, S_ROUTER_IN_DNAT, 50, "ip", "flags.loopback = 1; ct_dnat;"); } else { - /* For NAT on a distributed router, add flows to Ingress - * IP Routing table, Ingress ARP Resolution table, and - * Ingress Gateway Redirect Table that are not specific to a - * NAT rule. */ - - /* The highest priority IN_IP_ROUTING rule matches packets - * with REGBIT_NAT_REDIRECT (set in DNAT or UNSNAT stages), - * with action "ip.ttl--; next;". The IN_GW_REDIRECT table - * will take care of setting the outport. */ - ovn_lflow_add(lflows, od, S_ROUTER_IN_IP_ROUTING, 300, - REGBIT_NAT_REDIRECT" == 1", "ip.ttl--; next;"); - - /* The highest priority IN_ARP_RESOLVE rule matches packets - * with REGBIT_NAT_REDIRECT (set in DNAT or UNSNAT stages), - * then sets eth.dst to the distributed gateway port's - * ethernet address. */ - ds_clear(&actions); - ds_put_format(&actions, "eth.dst = %s; next;", - od->l3dgw_port->lrp_networks.ea_s); - ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 200, - REGBIT_NAT_REDIRECT" == 1", ds_cstr(&actions)); - - /* The highest priority IN_GW_REDIRECT rule redirects packets - * with REGBIT_NAT_REDIRECT (set in DNAT or UNSNAT stages) to - * the central instance of the l3dgw_port for NAT processing. */ - ds_clear(&actions); - ds_put_format(&actions, "outport = %s; next;", - od->l3redirect_port->json_key); - ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 200, - REGBIT_NAT_REDIRECT" == 1", ds_cstr(&actions)); + /* For NAT on a distributed router, add flows to Ingress + * ARP Resolution table, and Ingress Gateway Redirect Table + * that are not specific to a NAT rule. */ + for (size_t i = 0; i < od->n_l3dgw_ports; i++) { + struct ovn_port *l3dgw_port = od->l3dgw_ports[i]; + /* The highest priority IN_ARP_RESOLVE rule matches packets + * with REGBIT_NAT_REDIRECT (set in DNAT or UNSNAT stages), + * then sets eth.dst to the distributed gateway port's + * ethernet address. */ + ds_clear(&match); + ds_put_format(&match, REGBIT_NAT_REDIRECT" == 1 && " + "outport == %s", l3dgw_port->json_key); + ds_clear(&actions); + ds_put_format(&actions, "eth.dst = %s; next;", + l3dgw_port->lrp_networks.ea_s); + ovn_lflow_add(lflows, od, S_ROUTER_IN_ARP_RESOLVE, 200, + ds_cstr(&match), ds_cstr(&actions)); + + /* The highest priority IN_GW_REDIRECT rule redirects packets + * with REGBIT_NAT_REDIRECT (set in DNAT or UNSNAT stages) to + * the central instance of the l3dgw_port for NAT processing. */ + ds_clear(&actions); + ds_put_format(&actions, "outport = %s; next;", + l3dgw_port->l3redirect_port->json_key); + ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 200, + ds_cstr(&match), ds_cstr(&actions)); + } } /* Load balancing and packet defrag are only valid on @@ -4959,17 +4970,18 @@ build_lrouter_flows(struct hmap *datapaths, struct hmap *ports, if (!od->nbr) { continue; } - if (od->l3dgw_port && od->l3redirect_port) { + for (size_t i = 0; i < od->n_l3dgw_ports; i++) { + struct ovn_port *l3dgw_port = od->l3dgw_ports[i]; /* For traffic with outport == l3dgw_port, if the * packet did not match any higher priority redirect * rule, then the traffic is redirected to the central * instance of the l3dgw_port. */ ds_clear(&match); ds_put_format(&match, "outport == %s", - od->l3dgw_port->json_key); + l3dgw_port->json_key); ds_clear(&actions); ds_put_format(&actions, "outport = %s; next;", - od->l3redirect_port->json_key); + l3dgw_port->l3redirect_port->json_key); ovn_lflow_add(lflows, od, S_ROUTER_IN_GW_REDIRECT, 50, ds_cstr(&match), ds_cstr(&actions)); diff --git a/ovn/ovn-nb.xml b/ovn/ovn-nb.xml index c5ebbea..1286b1b 100644 --- a/ovn/ovn-nb.xml +++ b/ovn/ovn-nb.xml @@ -1112,8 +1112,7 @@

      If set, this indicates that this logical router port represents a distributed gateway port that connects this router to a logical - switch with a localnet port. There may be at most one such - logical router port on each logical router. + switch with a localnet port.

      @@ -1273,7 +1272,14 @@ - An IPv4 address. +

      + An IPv4 address. +

      + +

      + On distributed router, This address must be within the subnet of + the gateway port instance on the redirect-chassis. +