From patchwork Wed Jan 17 05:47:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Zhou X-Patchwork-Id: 1887253 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TFFKV4M1yz1yPg for ; Wed, 17 Jan 2024 16:48:14 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 20C7883B82; Wed, 17 Jan 2024 05:48:11 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 20C7883B82 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id R3mgxJ0Zh1Rw; Wed, 17 Jan 2024 05:48:09 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id 1AE7983A88; Wed, 17 Jan 2024 05:48:08 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 1AE7983A88 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 9BA7BC0DD6; Wed, 17 Jan 2024 05:48:05 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 60053C0037 for ; Wed, 17 Jan 2024 05:48:04 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 485DD41A32 for ; Wed, 17 Jan 2024 05:48:04 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 485DD41A32 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WRFyW7_hpcPy for ; Wed, 17 Jan 2024 05:48:03 +0000 (UTC) Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::225]) by smtp2.osuosl.org (Postfix) with ESMTPS id 9E952419FB for ; Wed, 17 Jan 2024 05:48:02 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 9E952419FB Received: by mail.gandi.net (Postfix) with ESMTPSA id 16A001C0006; Wed, 17 Jan 2024 05:47:59 +0000 (UTC) From: Han Zhou To: dev@openvswitch.org Date: Tue, 16 Jan 2024 21:47:43 -0800 Message-Id: <20240117054745.4027120-2-hzhou@ovn.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20240117054745.4027120-1-hzhou@ovn.org> References: <20240117054745.4027120-1-hzhou@ovn.org> MIME-Version: 1.0 X-GND-Sasl: hzhou@ovn.org Subject: [ovs-dev] [PATCH ovn 1/3] encaps: Refactor the naming related to tunnels. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Rename vars and structs to reflect the fact that there can be multiple tunnels for each individual chassis. Also update the documentation of external_ids:ovn-encap-ip. Signed-off-by: Han Zhou Acked-by: Ales Musil --- controller/encaps.c | 77 ++++++++++++++++----------------- controller/ovn-controller.8.xml | 3 +- 2 files changed, 40 insertions(+), 40 deletions(-) diff --git a/controller/encaps.c b/controller/encaps.c index b69d725843e9..1f6e667a606c 100644 --- a/controller/encaps.c +++ b/controller/encaps.c @@ -31,10 +31,12 @@ VLOG_DEFINE_THIS_MODULE(encaps); /* * Given there could be multiple tunnels with different IPs to the same - * chassis we annotate the ovn-chassis-id with - * OVN_MVTEP_CHASSISID_DELIM. + * chassis we annotate the external_ids:ovn-chassis-id in tunnel port with + * OVN_MVTEP_CHASSISID_DELIM. The external_id key + * "ovn-chassis-id" is kept for backward compatibility. */ #define OVN_MVTEP_CHASSISID_DELIM '@' +#define OVN_TUNNEL_ID "ovn-chassis-id" static char *current_br_int_name = NULL; @@ -55,8 +57,9 @@ encaps_register_ovs_idl(struct ovsdb_idl *ovs_idl) /* Enough context to create a new tunnel, using tunnel_add(). */ struct tunnel_ctx { - /* Maps from a chassis name to "struct chassis_node *". */ - struct shash chassis; + /* Maps from a tunnel-id (stored in external_ids:ovn-chassis-id) to + * "struct tunnel_node *". */ + struct shash tunnel; /* Names of all ports in the bridge, to allow checking uniqueness when * adding a new tunnel. */ @@ -68,7 +71,7 @@ struct tunnel_ctx { const struct sbrec_chassis *this_chassis; }; -struct chassis_node { +struct tunnel_node { const struct ovsrec_port *port; const struct ovsrec_bridge *bridge; }; @@ -104,7 +107,7 @@ encaps_tunnel_id_create(const char *chassis_id, const char *encap_ip) /* * Parses a 'tunnel_id' of the form . * If the 'chassis_id' argument is not NULL the function will allocate memory - * and store the chassis-id part of the tunnel-id at '*chassis_id'. + * and store the chassis_name part of the tunnel-id at '*chassis_id'. * If the 'encap_ip' argument is not NULL the function will allocate memory * and store the encapsulation IP part of the tunnel-id at '*encap_ip'. */ @@ -169,7 +172,7 @@ tunnel_add(struct tunnel_ctx *tc, const struct sbrec_sb_global *sbg, /* * Since a chassis may have multiple encap-ip, we can't just add the - * chassis name as as the "ovn-chassis-id" for the port; we use the + * chassis name as the OVN_TUNNEL_ID for the port; we use the * combination of the chassis_name and the encap-ip to identify * a specific tunnel to the chassis. */ @@ -260,25 +263,25 @@ tunnel_add(struct tunnel_ctx *tc, const struct sbrec_sb_global *sbg, } } - /* If there's an existing chassis record that does not need any change, + /* If there's an existing tunnel record that does not need any change, * keep it. Otherwise, create a new record (if there was an existing * record, the new record will supplant it and encaps_run() will delete * it). */ - struct chassis_node *chassis = shash_find_data(&tc->chassis, - tunnel_entry_id); - if (chassis - && chassis->port->n_interfaces == 1 - && !strcmp(chassis->port->interfaces[0]->type, encap->type) - && smap_equal(&chassis->port->interfaces[0]->options, &options)) { - shash_find_and_delete(&tc->chassis, tunnel_entry_id); - free(chassis); + struct tunnel_node *tunnel = shash_find_data(&tc->tunnel, + tunnel_entry_id); + if (tunnel + && tunnel->port->n_interfaces == 1 + && !strcmp(tunnel->port->interfaces[0]->type, encap->type) + && smap_equal(&tunnel->port->interfaces[0]->options, &options)) { + shash_find_and_delete(&tc->tunnel, tunnel_entry_id); + free(tunnel); goto exit; } /* Choose a name for the new port. If we're replacing an old port, reuse * its name, otherwise generate a new, unique name. */ - char *port_name = (chassis - ? xstrdup(chassis->port->name) + char *port_name = (tunnel + ? xstrdup(tunnel->port->name) : tunnel_create_name(tc, new_chassis_id)); if (!port_name) { VLOG_WARN("Unable to allocate unique name for '%s' tunnel", @@ -294,7 +297,7 @@ tunnel_add(struct tunnel_ctx *tc, const struct sbrec_sb_global *sbg, struct ovsrec_port *port = ovsrec_port_insert(tc->ovs_txn); ovsrec_port_set_name(port, port_name); ovsrec_port_set_interfaces(port, &iface, 1); - const struct smap id = SMAP_CONST1(&id, "ovn-chassis-id", tunnel_entry_id); + const struct smap id = SMAP_CONST1(&id, OVN_TUNNEL_ID, tunnel_entry_id); ovsrec_port_set_external_ids(port, &id); ovsrec_bridge_update_ports_addvalue(tc->br_int, port); @@ -394,7 +397,7 @@ clear_old_tunnels(const struct ovsrec_bridge *old_br_int, const char *prefix, { for (size_t i = 0; i < old_br_int->n_ports; i++) { const struct ovsrec_port *port = old_br_int->ports[i]; - const char *id = smap_get(&port->external_ids, "ovn-chassis-id"); + const char *id = smap_get(&port->external_ids, OVN_TUNNEL_ID); if (id && !strncmp(port->name, prefix, prefix_len)) { VLOG_DBG("Clearing old tunnel port \"%s\" (%s) from bridge " "\"%s\".", port->name, id, old_br_int->name); @@ -449,7 +452,7 @@ encaps_run(struct ovsdb_idl_txn *ovs_idl_txn, const struct sbrec_chassis *chassis_rec; struct tunnel_ctx tc = { - .chassis = SHASH_INITIALIZER(&tc.chassis), + .tunnel = SHASH_INITIALIZER(&tc.tunnel), .port_names = SSET_INITIALIZER(&tc.port_names), .br_int = br_int, .this_chassis = this_chassis, @@ -468,19 +471,15 @@ encaps_run(struct ovsdb_idl_txn *ovs_idl_txn, const struct ovsrec_port *port = br_int->ports[i]; sset_add(&tc.port_names, port->name); - /* - * note that the id here is not just the chassis name, but the - * combination of - */ - const char *id = smap_get(&port->external_ids, "ovn-chassis-id"); + const char *id = smap_get(&port->external_ids, OVN_TUNNEL_ID); if (id) { - if (!shash_find(&tc.chassis, id)) { - struct chassis_node *chassis = xzalloc(sizeof *chassis); - chassis->bridge = br_int; - chassis->port = port; - shash_add_assert(&tc.chassis, id, chassis); + if (!shash_find(&tc.tunnel, id)) { + struct tunnel_node *tunnel = xzalloc(sizeof *tunnel); + tunnel->bridge = br_int; + tunnel->port = port; + shash_add_assert(&tc.tunnel, id, tunnel); } else { - /* Duplicate port for ovn-chassis-id. Arbitrarily choose + /* Duplicate port for tunnel-id. Arbitrarily choose * to delete this one. */ ovsrec_bridge_update_ports_delvalue(br_int, port); } @@ -517,13 +516,13 @@ encaps_run(struct ovsdb_idl_txn *ovs_idl_txn, /* Delete any existing OVN tunnels that were not still around. */ struct shash_node *node; - SHASH_FOR_EACH_SAFE (node, &tc.chassis) { - struct chassis_node *chassis = node->data; - ovsrec_bridge_update_ports_delvalue(chassis->bridge, chassis->port); - shash_delete(&tc.chassis, node); - free(chassis); + SHASH_FOR_EACH_SAFE (node, &tc.tunnel) { + struct tunnel_node *tunnel = node->data; + ovsrec_bridge_update_ports_delvalue(tunnel->bridge, tunnel->port); + shash_delete(&tc.tunnel, node); + free(tunnel); } - shash_destroy(&tc.chassis); + shash_destroy(&tc.tunnel); sset_destroy(&tc.port_names); } @@ -542,7 +541,7 @@ encaps_cleanup(struct ovsdb_idl_txn *ovs_idl_txn, = xmalloc(sizeof *br_int->ports * br_int->n_ports); size_t n = 0; for (size_t i = 0; i < br_int->n_ports; i++) { - if (!smap_get(&br_int->ports[i]->external_ids, "ovn-chassis-id")) { + if (!smap_get(&br_int->ports[i]->external_ids, OVN_TUNNEL_ID)) { ports[n++] = br_int->ports[i]; } } diff --git a/controller/ovn-controller.8.xml b/controller/ovn-controller.8.xml index 735bc1221ab2..efa65e3fd927 100644 --- a/controller/ovn-controller.8.xml +++ b/controller/ovn-controller.8.xml @@ -178,7 +178,8 @@
The IP address that a chassis should use to connect to this node using encapsulation types specified by - external_ids:ovn-encap-type. + external_ids:ovn-encap-type. Multiple encapsulation IPs + may be specified with a comma-separated list.
external_ids:ovn-encap-df_default
From patchwork Wed Jan 17 05:47:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Zhou X-Patchwork-Id: 1887254 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TFFKW1zyfz23f0 for ; Wed, 17 Jan 2024 16:48:15 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 2EE3F6140A; Wed, 17 Jan 2024 05:48:13 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 2EE3F6140A X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bUmZFFkBxmVS; Wed, 17 Jan 2024 05:48:10 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 5CB03613F8; Wed, 17 Jan 2024 05:48:09 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 5CB03613F8 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 39E70C0DD9; Wed, 17 Jan 2024 05:48:07 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 01864C0DD9 for ; Wed, 17 Jan 2024 05:48:05 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id C9333613E6 for ; Wed, 17 Jan 2024 05:48:05 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org C9333613E6 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oO7TDYxCfole for ; Wed, 17 Jan 2024 05:48:04 +0000 (UTC) Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [IPv6:2001:4b98:dc4:8::225]) by smtp3.osuosl.org (Postfix) with ESMTPS id D5184613DE for ; Wed, 17 Jan 2024 05:48:03 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org D5184613DE Received: by mail.gandi.net (Postfix) with ESMTPSA id 43D2F1C0005; Wed, 17 Jan 2024 05:48:01 +0000 (UTC) From: Han Zhou To: dev@openvswitch.org Date: Tue, 16 Jan 2024 21:47:44 -0800 Message-Id: <20240117054745.4027120-3-hzhou@ovn.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20240117054745.4027120-1-hzhou@ovn.org> References: <20240117054745.4027120-1-hzhou@ovn.org> MIME-Version: 1.0 X-GND-Sasl: hzhou@ovn.org Cc: Lei Huang Subject: [ovs-dev] [PATCH ovn 2/3] encaps: Create separate tunnels for multiple local encap IPs. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" In commit dd527a283cd8, it created separate tunnels for different remote encap IPs of the same remote chassis, but when the current chassis is the one that has multiple encap IPs configured, it only uses the first encap IP. This patch creates separate tunnels taking into consideration the multiple encap IPs in current chassis and sets corresponding local_ip for each tunnel interface in such cases. Co-authored-by: Lei Huang Signed-off-by: Lei Huang Signed-off-by: Han Zhou Acked-by: Ales Musil --- controller/bfd.c | 4 +- controller/encaps.c | 158 ++++++++++++++++++++++------------------ controller/encaps.h | 9 ++- controller/lflow.c | 2 +- controller/local_data.c | 14 ++-- controller/local_data.h | 5 +- controller/physical.c | 28 +++---- controller/pinctrl.c | 2 +- tests/ovn-ipsec.at | 49 ------------- tests/ovn.at | 88 +++++++++++++++++----- 10 files changed, 192 insertions(+), 167 deletions(-) diff --git a/controller/bfd.c b/controller/bfd.c index cf011e382c6c..f24bfd063888 100644 --- a/controller/bfd.c +++ b/controller/bfd.c @@ -75,7 +75,7 @@ bfd_calculate_active_tunnels(const struct ovsrec_bridge *br_int, char *chassis_name = NULL; if (encaps_tunnel_id_parse(id, &chassis_name, - NULL)) { + NULL, NULL)) { if (!sset_contains(active_tunnels, chassis_name)) { sset_add(active_tunnels, chassis_name); @@ -204,7 +204,7 @@ bfd_run(const struct ovsrec_interface_table *interface_table, sset_add(&tunnels, port_name); - if (encaps_tunnel_id_parse(tunnel_id, &chassis_name, NULL)) { + if (encaps_tunnel_id_parse(tunnel_id, &chassis_name, NULL, NULL)) { if (sset_contains(&bfd_chassis, chassis_name)) { sset_add(&bfd_ifaces, port_name); } diff --git a/controller/encaps.c b/controller/encaps.c index 1f6e667a606c..28237f6191c8 100644 --- a/controller/encaps.c +++ b/controller/encaps.c @@ -32,10 +32,9 @@ VLOG_DEFINE_THIS_MODULE(encaps); /* * Given there could be multiple tunnels with different IPs to the same * chassis we annotate the external_ids:ovn-chassis-id in tunnel port with - * OVN_MVTEP_CHASSISID_DELIM. The external_id key + * @%. The external_id key * "ovn-chassis-id" is kept for backward compatibility. */ -#define OVN_MVTEP_CHASSISID_DELIM '@' #define OVN_TUNNEL_ID "ovn-chassis-id" static char *current_br_int_name = NULL; @@ -95,72 +94,93 @@ tunnel_create_name(struct tunnel_ctx *tc, const char *chassis_id) } /* - * Returns a tunnel-id of the form 'chassis_id'-delimiter-'encap_ip'. + * Returns a tunnel-id of the form chassis_id@remote_encap_ip%local_encap_ip. */ char * -encaps_tunnel_id_create(const char *chassis_id, const char *encap_ip) +encaps_tunnel_id_create(const char *chassis_id, const char *remote_encap_ip, + const char *local_encap_ip) { - return xasprintf("%s%c%s", chassis_id, OVN_MVTEP_CHASSISID_DELIM, - encap_ip); + return xasprintf("%s%c%s%c%s", chassis_id, '@', remote_encap_ip, + '%', local_encap_ip); } /* - * Parses a 'tunnel_id' of the form . + * Parses a 'tunnel_id' of the form @%. * If the 'chassis_id' argument is not NULL the function will allocate memory * and store the chassis_name part of the tunnel-id at '*chassis_id'. - * If the 'encap_ip' argument is not NULL the function will allocate memory - * and store the encapsulation IP part of the tunnel-id at '*encap_ip'. + * Same for remote_encap_ip and local_encap_ip. */ bool encaps_tunnel_id_parse(const char *tunnel_id, char **chassis_id, - char **encap_ip) + char **remote_encap_ip, char **local_encap_ip) { - /* Find the delimiter. Fail if there is no delimiter or if - * or is the empty string.*/ - const char *d = strchr(tunnel_id, OVN_MVTEP_CHASSISID_DELIM); + /* Find the @. Fail if there is no @ or if any part is empty. */ + const char *d = strchr(tunnel_id, '@'); if (d == tunnel_id || !d || !d[1]) { return false; } + /* Find the %. Fail if there is no % or if any part is empty. */ + const char *d2 = strchr(d + 1, '%'); + if (d2 == d + 1 || !d2 || !d2[1]) { + return false; + } + if (chassis_id) { *chassis_id = xmemdup0(tunnel_id, d - tunnel_id); } - if (encap_ip) { - *encap_ip = xstrdup(d + 1); + + if (remote_encap_ip) { + *remote_encap_ip = xmemdup0(d + 1, d2 - (d + 1)); + } + + if (local_encap_ip) { + *local_encap_ip = xstrdup(d2 + 1); } return true; } /* - * Returns true if 'tunnel_id' contains 'chassis_id' and, if specified, the - * given 'encap_ip'. Returns false otherwise. + * Returns true if 'tunnel_id' in the format + * @% + * contains 'chassis_id' and, if specified, the given 'remote_encap_ip' and + * 'local_encap_ip'. Returns false otherwise. */ bool encaps_tunnel_id_match(const char *tunnel_id, const char *chassis_id, - const char *encap_ip) + const char *remote_encap_ip, const char *local_encap_ip) { - while (*tunnel_id == *chassis_id) { - if (!*tunnel_id) { - /* 'tunnel_id' and 'chassis_id' are equal strings. This is a - * mismatch because 'tunnel_id' is missing the delimiter and IP. */ - return false; - } - tunnel_id++; - chassis_id++; + char *tokstr = xstrdup(tunnel_id); + char *saveptr = NULL; + bool ret = false; + + char *token_chassis = strtok_r(tokstr, "@", &saveptr); + if (!token_chassis || strcmp(token_chassis, chassis_id)) { + goto out; + } + + char *token_remote_ip = strtok_r(NULL, "%", &saveptr); + if (remote_encap_ip && + (!token_remote_ip || strcmp(token_remote_ip, remote_encap_ip))) { + goto out; } - /* We found the first byte that disagrees between 'tunnel_id' and - * 'chassis_id'. If we consumed all of 'chassis_id' and arrived at the - * delimiter in 'tunnel_id' (and if 'encap_ip' is correct, if it was - * supplied), it's a match. */ - return (*tunnel_id == OVN_MVTEP_CHASSISID_DELIM - && *chassis_id == '\0' - && (!encap_ip || !strcmp(tunnel_id + 1, encap_ip))); + char *token_local_ip = strtok_r(NULL, "", &saveptr); + if (local_encap_ip && + (!token_local_ip || strcmp(token_local_ip, local_encap_ip))) { + goto out; + } + + ret = true; +out: + free(tokstr); + return ret; } static void tunnel_add(struct tunnel_ctx *tc, const struct sbrec_sb_global *sbg, const char *new_chassis_id, const struct sbrec_encap *encap, + bool must_set_local_ip, const char *local_ip, const struct ovsrec_open_vswitch_table *ovs_table) { struct smap options = SMAP_INITIALIZER(&options); @@ -173,10 +193,11 @@ tunnel_add(struct tunnel_ctx *tc, const struct sbrec_sb_global *sbg, /* * Since a chassis may have multiple encap-ip, we can't just add the * chassis name as the OVN_TUNNEL_ID for the port; we use the - * combination of the chassis_name and the encap-ip to identify - * a specific tunnel to the chassis. + * combination of the chassis_name and the remote and local encap-ips to + * identify a specific tunnel to the remote chassis. */ - tunnel_entry_id = encaps_tunnel_id_create(new_chassis_id, encap->ip); + tunnel_entry_id = encaps_tunnel_id_create(new_chassis_id, encap->ip, + local_ip); if (csum && (!strcmp(csum, "true") || !strcmp(csum, "false"))) { smap_add(&options, "csum", csum); } @@ -187,7 +208,7 @@ tunnel_add(struct tunnel_ctx *tc, const struct sbrec_sb_global *sbg, const struct ovsrec_open_vswitch *cfg = ovsrec_open_vswitch_table_first(ovs_table); - bool set_local_ip = false; + bool set_local_ip = must_set_local_ip; if (cfg) { /* If the tos option is configured, get it */ const char *encap_tos = @@ -208,11 +229,13 @@ tunnel_add(struct tunnel_ctx *tc, const struct sbrec_sb_global *sbg, smap_add(&options, "df_default", encap_df); } - /* If ovn-set-local-ip option is configured, get it */ - set_local_ip = - get_chassis_external_id_value_bool( - &cfg->external_ids, tc->this_chassis->name, - "ovn-set-local-ip", false); + if (!set_local_ip) { + /* If ovn-set-local-ip option is configured, get it */ + set_local_ip = + get_chassis_external_id_value_bool( + &cfg->external_ids, tc->this_chassis->name, + "ovn-set-local-ip", false); + } } /* Add auth info if ipsec is enabled. */ @@ -237,30 +260,7 @@ tunnel_add(struct tunnel_ctx *tc, const struct sbrec_sb_global *sbg, } if (set_local_ip) { - const struct sbrec_chassis *this_chassis = tc->this_chassis; - const char *local_ip = NULL; - - /* Determine 'ovn-encap-ip' of the local chassis as this will be the - * tunnel port's 'local_ip'. We do not support the case in which - * 'ovn-encap-ip' holds multiple comma-delimited IP addresses. - */ - for (int i = 0; i < this_chassis->n_encaps; i++) { - if (local_ip && strcmp(local_ip, this_chassis->encaps[i]->ip)) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_ERR_RL(&rl, "ovn-encap-ip has been configured as a list. " - "This is unsupported for IPsec and explicit " - "local_ip configuration."); - /* No need to loop further as we know this condition has been - * hit */ - break; - } else { - local_ip = this_chassis->encaps[i]->ip; - } - } - - if (local_ip) { - smap_add(&options, "local_ip", local_ip); - } + smap_add(&options, "local_ip", local_ip); } /* If there's an existing tunnel record that does not need any change, @@ -362,9 +362,29 @@ chassis_tunnel_add(const struct sbrec_chassis *chassis_rec, if (tun_type != pref_type) { continue; } - tunnel_add(tc, sbg, chassis_rec->name, chassis_rec->encaps[i], - ovs_table); - tuncnt++; + + /* Check if need to pass the local ip. We always set local ip if there + * are multiple local IPs for the selected encap type. */ + int count = 0; + bool set_local_ip = false; + for (int j = 0; j < this_chassis->n_encaps; j++) { + if (pref_type == get_tunnel_type(this_chassis->encaps[j]->type) && + count++ > 0) { + set_local_ip = true; + break; + } + } + + for (int j = 0; j < this_chassis->n_encaps; j++) { + if (pref_type != get_tunnel_type(this_chassis->encaps[j]->type)) { + continue; + } + VLOG_DBG("tunnel_add: '%s', local ip: %s", chassis_rec->name, + this_chassis->encaps[j]->ip); + tunnel_add(tc, sbg, chassis_rec->name, chassis_rec->encaps[i], + set_local_ip, this_chassis->encaps[j]->ip, ovs_table); + tuncnt++; + } } return tuncnt; } diff --git a/controller/encaps.h b/controller/encaps.h index 3e58b3c82805..6f9891ee5907 100644 --- a/controller/encaps.h +++ b/controller/encaps.h @@ -41,11 +41,14 @@ void encaps_run(struct ovsdb_idl_txn *ovs_idl_txn, bool encaps_cleanup(struct ovsdb_idl_txn *ovs_idl_txn, const struct ovsrec_bridge *br_int); -char *encaps_tunnel_id_create(const char *chassis_id, const char *encap_ip); +char *encaps_tunnel_id_create(const char *chassis_id, + const char *remote_encap_ip, + const char *local_encap_ip); bool encaps_tunnel_id_parse(const char *tunnel_id, char **chassis_id, - char **encap_ip); + char **remote_encap_ip, char **local_encap_ip); bool encaps_tunnel_id_match(const char *tunnel_id, const char *chassis_id, - const char *encap_ip); + const char *remote_encap_ip, + const char *local_encap_ip); void encaps_destroy(void); diff --git a/controller/lflow.c b/controller/lflow.c index b0cf4253c482..c0cf0aa10646 100644 --- a/controller/lflow.c +++ b/controller/lflow.c @@ -164,7 +164,7 @@ tunnel_ofport_cb(const void *aux_, const char *port_name, ofp_port_t *ofport) } if (!get_chassis_tunnel_ofport(aux->chassis_tunnels, pb->chassis->name, - NULL, ofport)) { + ofport)) { return false; } diff --git a/controller/local_data.c b/controller/local_data.c index 3a7d3afebe0b..a9092783958f 100644 --- a/controller/local_data.c +++ b/controller/local_data.c @@ -388,7 +388,7 @@ local_nonvif_data_run(const struct ovsrec_bridge *br_int, "ovn-chassis-id"); if (tunnel_id && encaps_tunnel_id_match(tunnel_id, chassis_rec->name, - NULL)) { + NULL, NULL)) { continue; } @@ -439,7 +439,7 @@ local_nonvif_data_run(const struct ovsrec_bridge *br_int, char *hash_id = NULL; char *ip = NULL; - if (!encaps_tunnel_id_parse(tunnel_id, &hash_id, &ip)) { + if (!encaps_tunnel_id_parse(tunnel_id, &hash_id, &ip, NULL)) { continue; } struct chassis_tunnel *tun = xmalloc(sizeof *tun); @@ -477,11 +477,10 @@ local_nonvif_data_handle_ovs_iface_changes( bool get_chassis_tunnel_ofport(const struct hmap *chassis_tunnels, - const char *chassis_name, char *encap_ip, - ofp_port_t *ofport) + const char *chassis_name, ofp_port_t *ofport) { struct chassis_tunnel *tun = NULL; - tun = chassis_tunnel_find(chassis_tunnels, chassis_name, encap_ip); + tun = chassis_tunnel_find(chassis_tunnels, chassis_name, NULL, NULL); if (!tun) { return false; } @@ -515,7 +514,7 @@ chassis_tunnels_destroy(struct hmap *chassis_tunnels) */ struct chassis_tunnel * chassis_tunnel_find(const struct hmap *chassis_tunnels, const char *chassis_id, - char *encap_ip) + char *remote_encap_ip, char *local_encap_ip) { /* * If the specific encap_ip is given, look for the chassisid_ip entry, @@ -524,7 +523,8 @@ chassis_tunnel_find(const struct hmap *chassis_tunnels, const char *chassis_id, struct chassis_tunnel *tun = NULL; HMAP_FOR_EACH_WITH_HASH (tun, hmap_node, hash_string(chassis_id, 0), chassis_tunnels) { - if (encaps_tunnel_id_match(tun->chassis_id, chassis_id, encap_ip)) { + if (encaps_tunnel_id_match(tun->chassis_id, chassis_id, + remote_encap_ip, local_encap_ip)) { return tun; } } diff --git a/controller/local_data.h b/controller/local_data.h index f6d8f725f805..bab95bcc3824 100644 --- a/controller/local_data.h +++ b/controller/local_data.h @@ -149,10 +149,11 @@ bool local_nonvif_data_handle_ovs_iface_changes( struct chassis_tunnel *chassis_tunnel_find(const struct hmap *chassis_tunnels, const char *chassis_id, - char *encap_ip); + char *remote_encap_ip, + char *local_encap_ip); bool get_chassis_tunnel_ofport(const struct hmap *chassis_tunnels, - const char *chassis_name, char *encap_ip, + const char *chassis_name, ofp_port_t *ofport); void chassis_tunnels_destroy(struct hmap *chassis_tunnels); diff --git a/controller/physical.c b/controller/physical.c index eda0854410b6..e93bfbd2cffb 100644 --- a/controller/physical.c +++ b/controller/physical.c @@ -137,16 +137,18 @@ put_resubmit(uint8_t table_id, struct ofpbuf *ofpacts) * port. */ static struct chassis_tunnel * -get_port_binding_tun(const struct sbrec_encap *encap, +get_port_binding_tun(const struct sbrec_encap *remote_encap, + const struct sbrec_encap *local_encap, const struct sbrec_chassis *chassis, const struct hmap *chassis_tunnels) { struct chassis_tunnel *tun = NULL; - if (encap) { - tun = chassis_tunnel_find(chassis_tunnels, chassis->name, encap->ip); - } + tun = chassis_tunnel_find(chassis_tunnels, chassis->name, + remote_encap ? remote_encap->ip : NULL, + local_encap ? local_encap->ip : NULL); + if (!tun) { - tun = chassis_tunnel_find(chassis_tunnels, chassis->name, NULL); + tun = chassis_tunnel_find(chassis_tunnels, chassis->name, NULL, NULL); } return tun; } @@ -335,7 +337,7 @@ get_remote_tunnels(const struct sbrec_port_binding *binding, ovs_list_init(tunnels); if (binding->chassis && binding->chassis != chassis) { - tun = get_port_binding_tun(binding->encap, binding->chassis, + tun = get_port_binding_tun(binding->encap, NULL, binding->chassis, chassis_tunnels); if (!tun) { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); @@ -356,7 +358,7 @@ get_remote_tunnels(const struct sbrec_port_binding *binding, } const struct sbrec_encap *additional_encap; additional_encap = find_additional_encap_for_chassis(binding, chassis); - tun = get_port_binding_tun(additional_encap, + tun = get_port_binding_tun(additional_encap, NULL, binding->additional_chassis[i], chassis_tunnels); if (!tun) { @@ -432,10 +434,10 @@ put_remote_port_redirect_overlay_ha_remote( continue; } if (!tun) { - tun = chassis_tunnel_find(chassis_tunnels, ch->name, NULL); + tun = chassis_tunnel_find(chassis_tunnels, ch->name, NULL, NULL); } else { struct chassis_tunnel *chassis_tunnel = - chassis_tunnel_find(chassis_tunnels, ch->name, NULL); + chassis_tunnel_find(chassis_tunnels, ch->name, NULL, NULL); if (chassis_tunnel && tun->type != chassis_tunnel->type) { static struct vlog_rate_limit rl = @@ -469,7 +471,7 @@ put_remote_port_redirect_overlay_ha_remote( if (!ch) { continue; } - tun = chassis_tunnel_find(chassis_tunnels, ch->name, NULL); + tun = chassis_tunnel_find(chassis_tunnels, ch->name, NULL, NULL); if (!tun) { continue; } @@ -1930,7 +1932,7 @@ tunnel_to_chassis(enum mf_field_id mff_ovn_geneve, uint16_t outport, struct ofpbuf *remote_ofpacts) { const struct chassis_tunnel *tun - = chassis_tunnel_find(chassis_tunnels, chassis_name, NULL); + = chassis_tunnel_find(chassis_tunnels, chassis_name, NULL, NULL); if (!tun) { return; } @@ -1953,7 +1955,7 @@ fanout_to_chassis(enum mf_field_id mff_ovn_geneve, const struct chassis_tunnel *prev = NULL; SSET_FOR_EACH (chassis_name, remote_chassis) { const struct chassis_tunnel *tun - = chassis_tunnel_find(chassis_tunnels, chassis_name, NULL); + = chassis_tunnel_find(chassis_tunnels, chassis_name, NULL, NULL); if (!tun) { continue; } @@ -2492,7 +2494,7 @@ physical_run(struct physical_ctx *p_ctx, if (!binding->chassis || !encaps_tunnel_id_match(tun->chassis_id, - binding->chassis->name, NULL)) { + binding->chassis->name, NULL, NULL)) { continue; } diff --git a/controller/pinctrl.c b/controller/pinctrl.c index 4992eab0892b..4e35af0fb074 100644 --- a/controller/pinctrl.c +++ b/controller/pinctrl.c @@ -5780,7 +5780,7 @@ get_localnet_vifs_l3gwports( const char *tunnel_id = smap_get(&port_rec->external_ids, "ovn-chassis-id"); if (tunnel_id && - encaps_tunnel_id_match(tunnel_id, chassis->name, NULL)) { + encaps_tunnel_id_match(tunnel_id, chassis->name, NULL, NULL)) { continue; } const char *localnet = smap_get(&port_rec->external_ids, diff --git a/tests/ovn-ipsec.at b/tests/ovn-ipsec.at index f8df8d60e4b3..7579124db2e1 100644 --- a/tests/ovn-ipsec.at +++ b/tests/ovn-ipsec.at @@ -58,52 +58,3 @@ AT_CHECK([as hv1 ovs-vsctl get Interface ovn-hv2-0 options:remote_name | tr -d ' AT_CHECK([as hv1 ovs-vsctl get Interface ovn-hv2-0 options:ipsec_encapsulation | tr -d '\n'], [0], [yes]) AT_CLEANUP - -AT_SETUP([ipsec -- unsupported multiple ovn-encap-ip values]) -ovn_start - -# Configure the Northbound database -ovn-nbctl ls-add lsw0 - -ovn-nbctl lsp-add lsw0 lp1 -ovn-nbctl lsp-set-addresses lp1 "f0:00:00:00:00:01 10.1.1.1" - -ovn-nbctl lsp-add lsw0 lp2 -ovn-nbctl lsp-set-addresses lp2 "f0:00:00:00:00:02 10.1.1.2" - -net_add n1 # Network to connect hv1 and hv2 - -# Enable IPsec -ovn-nbctl set nb_global . ipsec=true - -# Create hypervisor hv1 connected to n1 -sim_add hv1 -as hv1 -ovs-vsctl add-br br-phys -ovn_attach n1 br-phys 192.168.0.1 -ovs-vsctl add-port br-int vif1 -- set Interface vif1 external-ids:iface-id=lp1 -ovs-vsctl \ - -- set Open_vSwitch . external-ids:system-id=hv1 \ - -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \ - -- set Open_vSwitch . external-ids:ovn-encap-ip="192.168.0.1, 192.169.0.1" \ - -- set Open_vSwitch . other_config:certificate=dummy-cert.pem \ - -- set Open_vSwitch . other_config:private_key=dummy-privkey.pem \ - -- set Open_vSwitch . other_config:ca_cert=dummy-cacert.pem - -# Create hypervisor hv2 connected to n1 -sim_add hv2 -as hv2 -ovs-vsctl add-br br-phys -ovn_attach n1 br-phys 192.168.0.2 -ovs-vsctl add-port br-int vif2 -- set Interface vif2 external-ids:iface-id=lp2 -ovs-vsctl \ - -- set Open_vSwitch . external-ids:system-id=hv2 \ - -- set Open_vSwitch . external-ids:ovn-encap-type=geneve \ - -- set Open_vSwitch . external-ids:ovn-encap-ip="192.168.0.2, 192.169.0.2" \ - -- set Open_vSwitch . other_config:certificate=dummy-cert.pem \ - -- set Open_vSwitch . other_config:private_key=dummy-privkey.pem \ - -- set Open_vSwitch . other_config:ca_cert=dummy-cacert.pem - -OVS_WAIT_UNTIL([grep "ovn-encap-ip has been configured as a list. This is unsupported for IPsec." hv1/ovn-controller.log]) - -AT_CLEANUP diff --git a/tests/ovn.at b/tests/ovn.at index 2dd46fd79452..243fe0b8246c 100644 --- a/tests/ovn.at +++ b/tests/ovn.at @@ -30333,6 +30333,54 @@ OVN_CLEANUP([hv1]) AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD([ +AT_SETUP([multiple encap ips tunnel creation]) +ovn_start +net_add n1 + +# 2 HVs, each with 2 encap-ips. +for i in 1 2; do + sim_add hv$i + as hv$i + ovs-vsctl add-br br-phys-$j + ovn_attach n1 br-phys-$j 192.168.0.${i}1 + ovs-vsctl set open . external_ids:ovn-encap-ip=192.168.0.${i}1,192.168.0.${i}2 +done + +check ovn-nbctl --wait=hv sync + +check_tunnel_port() { + local hv=$1 + local br=$2 + local id=$3 + + as $hv + OVS_WAIT_UNTIL([ + test "$(ovs-vsctl --format=table --no-headings find port external_ids:ovn-chassis-id="$id" | wc -l)" = "1" + ]) + local tunnel_id=$(ovs-vsctl --bare --columns _uuid find port external_ids:ovn-chassis-id="$id") + AT_CHECK([ovs-vsctl --bare --columns ports find bridge name="$br" | grep -q "$tunnel_id"]) +} + +# Check that both chassis have tunnels. +# 'tunnel_id' in the format: +# @% +check_tunnel_port hv1 br-int hv2@192.168.0.21%192.168.0.11 +check_tunnel_port hv1 br-int hv2@192.168.0.22%192.168.0.11 +check_tunnel_port hv1 br-int hv2@192.168.0.21%192.168.0.12 +check_tunnel_port hv1 br-int hv2@192.168.0.22%192.168.0.12 + +check_tunnel_port hv2 br-int hv1@192.168.0.11%192.168.0.21 +check_tunnel_port hv2 br-int hv1@192.168.0.12%192.168.0.21 +check_tunnel_port hv2 br-int hv1@192.168.0.11%192.168.0.22 +check_tunnel_port hv2 br-int hv1@192.168.0.12%192.168.0.22 + +OVN_CLEANUP([hv1],[hv2]) +AT_CLEANUP +]) + + OVN_FOR_EACH_NORTHD([ AT_SETUP([Load Balancer LS hairpin OF flows]) ovn_start @@ -36174,14 +36222,14 @@ check_tunnel_port() { } # Check that both chassis have tunnel -check_tunnel_port hv1 br-int hv2@192.168.0.2 -check_tunnel_port hv2 br-int hv1@192.168.0.1 +check_tunnel_port hv1 br-int hv2@192.168.0.2%192.168.0.1 +check_tunnel_port hv2 br-int hv1@192.168.0.1%192.168.0.2 # Stop ovn-controller on hv1 check as hv1 ovn-appctl -t ovn-controller exit --restart # The tunnel should remain intact -check_tunnel_port hv1 br-int hv2@192.168.0.2 +check_tunnel_port hv1 br-int hv2@192.168.0.2%192.168.0.1 # Change the bridge to br-int1 on hv1 as hv1 @@ -36191,8 +36239,8 @@ start_daemon ovn-controller --verbose="encaps:dbg" check ovn-nbctl --wait=hv sync # Check that the tunnel was created on br-int1 instead -check_tunnel_port hv1 br-int1 hv2@192.168.0.2 -check grep -q "Clearing old tunnel port \"ovn-hv2-0\" (hv2@192.168.0.2) from bridge \"br-int\"" hv1/ovn-controller.log +check_tunnel_port hv1 br-int1 hv2@192.168.0.2%192.168.0.1 +check grep -q "Clearing old tunnel port \"ovn-hv2-0\" (hv2@192.168.0.2%192.168.0.1) from bridge \"br-int\"" hv1/ovn-controller.log # Change the bridge to br-int1 on hv2 as hv2 @@ -36203,21 +36251,21 @@ check ovn-nbctl --wait=hv sync # Check that the tunnel was created on br-int1 instead -check_tunnel_port hv2 br-int1 hv1@192.168.0.1 -check grep -q "Clearing old tunnel port \"ovn-hv1-0\" (hv1@192.168.0.1) from bridge \"br-int\"" hv2/ovn-controller.log +check_tunnel_port hv2 br-int1 hv1@192.168.0.1%192.168.0.2 +check grep -q "Clearing old tunnel port \"ovn-hv1-0\" (hv1@192.168.0.1%192.168.0.2) from bridge \"br-int\"" hv2/ovn-controller.log # Stop ovn-controller on hv1 check as hv1 ovn-appctl -t ovn-controller exit --restart # The tunnel should remain intact -check_tunnel_port hv1 br-int1 hv2@192.168.0.2 -prev_id=$(ovs-vsctl --bare --columns _uuid find port external_ids:ovn-chassis-id="hv2@192.168.0.2") +check_tunnel_port hv1 br-int1 hv2@192.168.0.2%192.168.0.1 +prev_id=$(ovs-vsctl --bare --columns _uuid find port external_ids:ovn-chassis-id="hv2@192.168.0.2%192.168.0.1") # Start the controller again start_daemon ovn-controller --verbose="encaps:dbg" check ovn-nbctl --wait=hv sync -check_tunnel_port hv1 br-int1 hv2@192.168.0.2 -current_id=$(ovs-vsctl --bare --columns _uuid find port external_ids:ovn-chassis-id="hv2@192.168.0.2") +check_tunnel_port hv1 br-int1 hv2@192.168.0.2%192.168.0.1 +current_id=$(ovs-vsctl --bare --columns _uuid find port external_ids:ovn-chassis-id="hv2@192.168.0.2%192.168.0.1") # The tunnel should be the same after restart check test "$current_id" = "$prev_id" @@ -36276,10 +36324,10 @@ check_tunnel_port() { AT_CHECK([ovs-vsctl --bare --columns ports find bridge name="$br" | grep -q "$tunnel_id"]) } -check_tunnel_port hv1 br-int hv2@192.168.0.2 -check_tunnel_port hv1 br-int-2 hv1@192.168.0.1 -prev_id1=$(ovs-vsctl --bare --columns _uuid find port external_ids:ovn-chassis-id="hv1@192.168.0.1") -prev_id2=$(ovs-vsctl --bare --columns _uuid find port external_ids:ovn-chassis-id="hv2@192.168.0.2") +check_tunnel_port hv1 br-int hv2@192.168.0.2%192.168.0.1 +check_tunnel_port hv1 br-int-2 hv1@192.168.0.1%192.168.0.2 +prev_id1=$(ovs-vsctl --bare --columns _uuid find port external_ids:ovn-chassis-id="hv1@192.168.0.1%192.168.0.2") +prev_id2=$(ovs-vsctl --bare --columns _uuid find port external_ids:ovn-chassis-id="hv2@192.168.0.2%192.168.0.1") # The hv2 is running we can remove the override file rm -f ${OVN_SYSCONFDIR}/system-id-override @@ -36300,13 +36348,13 @@ start_daemon ovn-controller --verbose="encaps:dbg" \ check ovn-nbctl --wait=hv sync -check_tunnel_port hv1 br-int hv2@192.168.0.2 -check_tunnel_port hv1 br-int-2 hv1@192.168.0.1 -current_id1=$(ovs-vsctl --bare --columns _uuid find port external_ids:ovn-chassis-id="hv1@192.168.0.1") -current_id2=$(ovs-vsctl --bare --columns _uuid find port external_ids:ovn-chassis-id="hv2@192.168.0.2") +check_tunnel_port hv1 br-int hv2@192.168.0.2%192.168.0.1 +check_tunnel_port hv1 br-int-2 hv1@192.168.0.1%192.168.0.2 +current_id1=$(ovs-vsctl --bare --columns _uuid find port external_ids:ovn-chassis-id="hv1@192.168.0.1%192.168.0.2") +current_id2=$(ovs-vsctl --bare --columns _uuid find port external_ids:ovn-chassis-id="hv2@192.168.0.2%192.168.0.1") # Check that restart of hv1 ovn-controller did not interfere with hv2 -AT_CHECK([grep -q "Clearing old tunnel port \"ovn0-hv1-0\" (hv1@192.168.0.1) from bridge \"br-int-2\"" hv1/ovn-controller.log], [1]) +AT_CHECK([grep -q "Clearing old tunnel port \"ovn0-hv1-0\" (hv1@192.168.0.1%192.168.0.2) from bridge \"br-int-2\"" hv1/ovn-controller.log], [1]) check test "$current_id1" = "$prev_id1" check test "$current_id2" = "$prev_id2" From patchwork Wed Jan 17 05:47:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Han Zhou X-Patchwork-Id: 1887255 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4TFFKd3Bncz1yPg for ; Wed, 17 Jan 2024 16:48:21 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 2CEA983CF2; Wed, 17 Jan 2024 05:48:19 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 2CEA983CF2 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1eeZALVAV1Pp; Wed, 17 Jan 2024 05:48:14 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 6500883BB1; Wed, 17 Jan 2024 05:48:12 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 6500883BB1 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 78953C0DE1; Wed, 17 Jan 2024 05:48:10 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id C3B90C007C for ; Wed, 17 Jan 2024 05:48:08 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 8D7F64024C for ; Wed, 17 Jan 2024 05:48:08 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 8D7F64024C X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2NsJPkMZR4Qw for ; Wed, 17 Jan 2024 05:48:06 +0000 (UTC) Received: from relay5-d.mail.gandi.net (relay5-d.mail.gandi.net [217.70.183.197]) by smtp4.osuosl.org (Postfix) with ESMTPS id 5F63F40234 for ; Wed, 17 Jan 2024 05:48:05 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 5F63F40234 Received: by mail.gandi.net (Postfix) with ESMTPSA id DD2F11C0007; Wed, 17 Jan 2024 05:48:02 +0000 (UTC) From: Han Zhou To: dev@openvswitch.org Date: Tue, 16 Jan 2024 21:47:45 -0800 Message-Id: <20240117054745.4027120-4-hzhou@ovn.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20240117054745.4027120-1-hzhou@ovn.org> References: <20240117054745.4027120-1-hzhou@ovn.org> MIME-Version: 1.0 X-GND-Spam-Score: 300 X-GND-Status: SPAM X-GND-Sasl: hzhou@ovn.org Cc: Lei Huang Subject: [ovs-dev] [PATCH ovn 3/3] ovn-controller: Support VIF-based local encap IPs selection. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Commit dd527a283cd8 partially supported multiple encap IPs. It supported remote encap IP selection based on the destination VIF's encap_ip configuration. This patch adds the support for selecting local encap IP based on the source VIF's encap_ip configuration. Co-authored-by: Lei Huang Signed-off-by: Lei Huang Signed-off-by: Han Zhou Acked-by: Ales Musil --- NEWS | 3 + controller/chassis.c | 2 +- controller/local_data.c | 2 +- controller/local_data.h | 2 +- controller/ovn-controller.8.xml | 30 ++++++- controller/ovn-controller.c | 49 ++++++++++++ controller/physical.c | 134 ++++++++++++++++++++++---------- controller/physical.h | 2 + include/ovn/logical-fields.h | 4 +- ovn-architecture.7.xml | 18 ++++- tests/ovn.at | 51 +++++++++++- 11 files changed, 243 insertions(+), 54 deletions(-) diff --git a/NEWS b/NEWS index 5f267b4c64cc..5a3eed608617 100644 --- a/NEWS +++ b/NEWS @@ -14,6 +14,9 @@ Post v23.09.0 - ovn-northd-ddlog has been removed. - A new LSP option "enable_router_port_acl" has been added to enable conntrack for the router port whose peer is l3dgw_port if set it true. + - Support selecting encapsulation IP based on the source/destination VIF's + settting. See ovn-controller(8) 'external_ids:ovn-encap-ip' for more + details. OVN v23.09.0 - 15 Sep 2023 -------------------------- diff --git a/controller/chassis.c b/controller/chassis.c index a6f13ccc42d5..55f2beb37674 100644 --- a/controller/chassis.c +++ b/controller/chassis.c @@ -61,7 +61,7 @@ struct ovs_chassis_cfg { /* Set of encap types parsed from the 'ovn-encap-type' external-id. */ struct sset encap_type_set; - /* Set of encap IPs parsed from the 'ovn-encap-type' external-id. */ + /* Set of encap IPs parsed from the 'ovn-encap-ip' external-id. */ struct sset encap_ip_set; /* Interface type list formatted in the OVN-SB Chassis required format. */ struct ds iface_types; diff --git a/controller/local_data.c b/controller/local_data.c index a9092783958f..8606414f8728 100644 --- a/controller/local_data.c +++ b/controller/local_data.c @@ -514,7 +514,7 @@ chassis_tunnels_destroy(struct hmap *chassis_tunnels) */ struct chassis_tunnel * chassis_tunnel_find(const struct hmap *chassis_tunnels, const char *chassis_id, - char *remote_encap_ip, char *local_encap_ip) + char *remote_encap_ip, const char *local_encap_ip) { /* * If the specific encap_ip is given, look for the chassisid_ip entry, diff --git a/controller/local_data.h b/controller/local_data.h index bab95bcc3824..ca3905bd20e6 100644 --- a/controller/local_data.h +++ b/controller/local_data.h @@ -150,7 +150,7 @@ bool local_nonvif_data_handle_ovs_iface_changes( struct chassis_tunnel *chassis_tunnel_find(const struct hmap *chassis_tunnels, const char *chassis_id, char *remote_encap_ip, - char *local_encap_ip); + const char *local_encap_ip); bool get_chassis_tunnel_ofport(const struct hmap *chassis_tunnels, const char *chassis_name, diff --git a/controller/ovn-controller.8.xml b/controller/ovn-controller.8.xml index efa65e3fd927..5ebef048d721 100644 --- a/controller/ovn-controller.8.xml +++ b/controller/ovn-controller.8.xml @@ -176,10 +176,32 @@
external_ids:ovn-encap-ip
- The IP address that a chassis should use to connect to this node - using encapsulation types specified by - external_ids:ovn-encap-type. Multiple encapsulation IPs - may be specified with a comma-separated list. +

+ The IP address that a chassis should use to connect to this node + using encapsulation types specified by + external_ids:ovn-encap-type. Multiple encapsulation IPs + may be specified with a comma-separated list. +

+

+ In scenarios where multiple encapsulation IPs are present, distinct + tunnels are established for each remote chassis. These tunnels are + differentiated by setting unique options:local_ip and + options:remote_ip values in the tunnel interface. When + transmitting a packet to a remote chassis, the selection of local_ip + is guided by the Interface:external_ids:encap-ip from + the local OVSDB, corresponding to the VIF originating the packet, if + specified. The Interface:external_ids:encap-ip setting + of the VIF is also populated to the Port_Binding + table in the OVN SB database via the encap column. + Consequently, when a remote chassis needs to send a packet to a + port-binding associated with this VIF, it utilizes the tunnel with + the appropriate options:remote_ip that matches the + ip in Port_Binding:encap. This mechanism + is particularly beneficial for chassis with multiple physical + interfaces designated for tunneling, where each interface is + optimized for handling specific traffic associated with particular + VIFs. +

external_ids:ovn-encap-df_default
diff --git a/controller/ovn-controller.c b/controller/ovn-controller.c index 856e5e270822..4ab57fe970fe 100644 --- a/controller/ovn-controller.c +++ b/controller/ovn-controller.c @@ -4521,6 +4521,31 @@ struct ed_type_pflow_output { struct physical_debug debug; }; +static void +parse_encap_ips(const struct ovsrec_open_vswitch_table *ovs_table, + size_t *n_encap_ips, const char * **encap_ips) +{ + const struct ovsrec_open_vswitch *cfg = + ovsrec_open_vswitch_table_first(ovs_table); + const char *chassis_id = get_ovs_chassis_id(ovs_table); + const char *encap_ips_str = + get_chassis_external_id_value(&cfg->external_ids, chassis_id, + "ovn-encap-ip", NULL); + struct sset encap_ip_set; + sset_init(&encap_ip_set); + sset_from_delimited_string(&encap_ip_set, encap_ips_str, ","); + + /* Sort the ips so that their index is deterministic. */ + *encap_ips = sset_sort(&encap_ip_set); + + /* Copy the strings so that we can destroy the sset. */ + for (size_t i = 0; (*encap_ips)[i]; i++) { + (*encap_ips)[i] = xstrdup((*encap_ips)[i]); + } + *n_encap_ips = sset_count(&encap_ip_set); + sset_destroy(&encap_ip_set); +} + static void init_physical_ctx(struct engine_node *node, struct ed_type_runtime_data *rt_data, struct ed_type_non_vif_data *non_vif_data, @@ -4572,6 +4597,7 @@ static void init_physical_ctx(struct engine_node *node, engine_get_input_data("ct_zones", node); struct simap *ct_zones = &ct_zones_data->current; + parse_encap_ips(ovs_table, &p_ctx->n_encap_ips, &p_ctx->encap_ips); p_ctx->sbrec_port_binding_by_name = sbrec_port_binding_by_name; p_ctx->sbrec_port_binding_by_datapath = sbrec_port_binding_by_datapath; p_ctx->port_binding_table = port_binding_table; @@ -4589,12 +4615,23 @@ static void init_physical_ctx(struct engine_node *node, p_ctx->patch_ofports = &non_vif_data->patch_ofports; p_ctx->chassis_tunnels = &non_vif_data->chassis_tunnels; + + struct controller_engine_ctx *ctrl_ctx = engine_get_context()->client_ctx; p_ctx->if_mgr = ctrl_ctx->if_mgr; pflow_output_get_debug(node, &p_ctx->debug); } +static void +destroy_physical_ctx(struct physical_ctx *p_ctx) +{ + for (size_t i = 0; i < p_ctx->n_encap_ips; i++) { + free((char *)(p_ctx->encap_ips[i])); + } + free(p_ctx->encap_ips); +} + static void * en_pflow_output_init(struct engine_node *node OVS_UNUSED, struct engine_arg *arg OVS_UNUSED) @@ -4631,6 +4668,7 @@ en_pflow_output_run(struct engine_node *node, void *data) struct physical_ctx p_ctx; init_physical_ctx(node, rt_data, non_vif_data, &p_ctx); physical_run(&p_ctx, pflow_table); + destroy_physical_ctx(&p_ctx); engine_set_node_state(node, EN_UPDATED); } @@ -4675,6 +4713,7 @@ pflow_output_if_status_mgr_handler(struct engine_node *node, bool removed = sbrec_port_binding_is_deleted(binding); if (!physical_handle_flows_for_lport(binding, removed, &p_ctx, &pfo->flow_table)) { + destroy_physical_ctx(&p_ctx); return false; } } @@ -4684,11 +4723,13 @@ pflow_output_if_status_mgr_handler(struct engine_node *node, bool removed = sbrec_port_binding_is_deleted(pb); if (!physical_handle_flows_for_lport(pb, removed, &p_ctx, &pfo->flow_table)) { + destroy_physical_ctx(&p_ctx); return false; } } engine_set_node_state(node, EN_UPDATED); } + destroy_physical_ctx(&p_ctx); return true; } @@ -4715,11 +4756,13 @@ pflow_output_sb_port_binding_handler(struct engine_node *node, bool removed = sbrec_port_binding_is_deleted(pb); if (!physical_handle_flows_for_lport(pb, removed, &p_ctx, &pfo->flow_table)) { + destroy_physical_ctx(&p_ctx); return false; } } engine_set_node_state(node, EN_UPDATED); + destroy_physical_ctx(&p_ctx); return true; } @@ -4739,6 +4782,7 @@ pflow_output_sb_multicast_group_handler(struct engine_node *node, void *data) physical_handle_mc_group_changes(&p_ctx, &pfo->flow_table); engine_set_node_state(node, EN_UPDATED); + destroy_physical_ctx(&p_ctx); return true; } @@ -4771,6 +4815,7 @@ pflow_output_runtime_data_handler(struct engine_node *node, void *data) if (tdp->tracked_type != TRACKED_RESOURCE_UPDATED) { /* Fall back to full recompute when a local datapath * is added or deleted. */ + destroy_physical_ctx(&p_ctx); return false; } @@ -4781,12 +4826,14 @@ pflow_output_runtime_data_handler(struct engine_node *node, void *data) lport->tracked_type == TRACKED_RESOURCE_REMOVED ? true: false; if (!physical_handle_flows_for_lport(lport->pb, removed, &p_ctx, &pfo->flow_table)) { + destroy_physical_ctx(&p_ctx); return false; } } } engine_set_node_state(node, EN_UPDATED); + destroy_physical_ctx(&p_ctx); return true; } @@ -4843,12 +4890,14 @@ pflow_output_activated_ports_handler(struct engine_node *node, void *data) if (pb) { if (!physical_handle_flows_for_lport(pb, false, &p_ctx, &pfo->flow_table)) { + destroy_physical_ctx(&p_ctx); return false; } tag_port_as_activated_in_engine(pp); } } engine_set_node_state(node, EN_UPDATED); + destroy_physical_ctx(&p_ctx); return true; } diff --git a/controller/physical.c b/controller/physical.c index e93bfbd2cffb..c192aed751d5 100644 --- a/controller/physical.c +++ b/controller/physical.c @@ -71,6 +71,8 @@ struct tunnel { static void load_logical_ingress_metadata(const struct sbrec_port_binding *binding, const struct zone_ids *zone_ids, + size_t n_encap_ips, + const char **encap_ips, struct ofpbuf *ofpacts_p); static int64_t get_vxlan_port_key(int64_t port_key); @@ -138,14 +140,14 @@ put_resubmit(uint8_t table_id, struct ofpbuf *ofpacts) */ static struct chassis_tunnel * get_port_binding_tun(const struct sbrec_encap *remote_encap, - const struct sbrec_encap *local_encap, const struct sbrec_chassis *chassis, - const struct hmap *chassis_tunnels) + const struct hmap *chassis_tunnels, + const char *local_encap_ip) { struct chassis_tunnel *tun = NULL; tun = chassis_tunnel_find(chassis_tunnels, chassis->name, remote_encap ? remote_encap->ip : NULL, - local_encap ? local_encap->ip : NULL); + local_encap_ip); if (!tun) { tun = chassis_tunnel_find(chassis_tunnels, chassis->name, NULL, NULL); @@ -329,7 +331,8 @@ find_additional_encap_for_chassis(const struct sbrec_port_binding *pb, static struct ovs_list * get_remote_tunnels(const struct sbrec_port_binding *binding, const struct sbrec_chassis *chassis, - const struct hmap *chassis_tunnels) + const struct hmap *chassis_tunnels, + const char *local_encap_ip) { const struct chassis_tunnel *tun; @@ -337,8 +340,8 @@ get_remote_tunnels(const struct sbrec_port_binding *binding, ovs_list_init(tunnels); if (binding->chassis && binding->chassis != chassis) { - tun = get_port_binding_tun(binding->encap, NULL, binding->chassis, - chassis_tunnels); + tun = get_port_binding_tun(binding->encap, binding->chassis, + chassis_tunnels, local_encap_ip); if (!tun) { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); VLOG_WARN_RL( @@ -358,9 +361,9 @@ get_remote_tunnels(const struct sbrec_port_binding *binding, } const struct sbrec_encap *additional_encap; additional_encap = find_additional_encap_for_chassis(binding, chassis); - tun = get_port_binding_tun(additional_encap, NULL, + tun = get_port_binding_tun(additional_encap, binding->additional_chassis[i], - chassis_tunnels); + chassis_tunnels, local_encap_ip); if (!tun) { static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); VLOG_WARN_RL( @@ -384,36 +387,48 @@ put_remote_port_redirect_overlay(const struct sbrec_port_binding *binding, struct ofpbuf *ofpacts_p, const struct sbrec_chassis *chassis, const struct hmap *chassis_tunnels, + size_t n_encap_ips, + const char **encap_ips, struct ovn_desired_flow_table *flow_table) { /* Setup encapsulation */ - struct ovs_list *tuns = get_remote_tunnels(binding, chassis, - chassis_tunnels); - if (!ovs_list_is_empty(tuns)) { - bool is_vtep_port = !strcmp(binding->type, "vtep"); - /* rewrite MFF_IN_PORT to bypass OpenFlow loopback check for ARP/ND - * responder in L3 networks. */ - if (is_vtep_port) { - put_load(ofp_to_u16(OFPP_NONE), MFF_IN_PORT, 0, 16, ofpacts_p); - } + for (size_t i = 0; i < n_encap_ips; i++) { + struct ofpbuf *ofpacts_clone = ofpbuf_clone(ofpacts_p); + + match_set_reg_masked(match, MFF_LOG_ENCAP_ID - MFF_REG0, i << 16, + (uint32_t) 0xFFFF << 16); + struct ovs_list *tuns = get_remote_tunnels(binding, chassis, + chassis_tunnels, + encap_ips[i]); + if (!ovs_list_is_empty(tuns)) { + bool is_vtep_port = !strcmp(binding->type, "vtep"); + /* rewrite MFF_IN_PORT to bypass OpenFlow loopback check for ARP/ND + * responder in L3 networks. */ + if (is_vtep_port) { + put_load(ofp_to_u16(OFPP_NONE), MFF_IN_PORT, 0, 16, + ofpacts_clone); + } - struct tunnel *tun; - LIST_FOR_EACH (tun, list_node, tuns) { - put_encapsulation(mff_ovn_geneve, tun->tun, - binding->datapath, port_key, is_vtep_port, - ofpacts_p); - ofpact_put_OUTPUT(ofpacts_p)->port = tun->tun->ofport; + struct tunnel *tun; + LIST_FOR_EACH (tun, list_node, tuns) { + put_encapsulation(mff_ovn_geneve, tun->tun, + binding->datapath, port_key, is_vtep_port, + ofpacts_clone); + ofpact_put_OUTPUT(ofpacts_clone)->port = tun->tun->ofport; + } + put_resubmit(OFTABLE_LOCAL_OUTPUT, ofpacts_clone); + ofctrl_add_flow(flow_table, OFTABLE_REMOTE_OUTPUT, 100, + binding->header_.uuid.parts[0], match, + ofpacts_clone, &binding->header_.uuid); } - put_resubmit(OFTABLE_LOCAL_OUTPUT, ofpacts_p); - ofctrl_add_flow(flow_table, OFTABLE_REMOTE_OUTPUT, 100, - binding->header_.uuid.parts[0], match, ofpacts_p, - &binding->header_.uuid); - } - struct tunnel *tun_elem; - LIST_FOR_EACH_POP (tun_elem, list_node, tuns) { - free(tun_elem); + struct tunnel *tun_elem; + LIST_FOR_EACH_POP (tun_elem, list_node, tuns) { + free(tun_elem); + } + free(tuns); + + ofpbuf_delete(ofpacts_clone); } - free(tuns); } static void @@ -673,7 +688,8 @@ put_replace_chassis_mac_flows(const struct simap *ct_zones, if (tag) { ofpact_put_STRIP_VLAN(ofpacts_p); } - load_logical_ingress_metadata(localnet_port, &zone_ids, ofpacts_p); + load_logical_ingress_metadata(localnet_port, &zone_ids, 0, NULL, + ofpacts_p); replace_mac = ofpact_put_SET_ETH_SRC(ofpacts_p); replace_mac->mac = router_port_mac; @@ -853,7 +869,7 @@ put_zones_ofpacts(const struct zone_ids *zone_ids, struct ofpbuf *ofpacts_p) { if (zone_ids) { if (zone_ids->ct) { - put_load(zone_ids->ct, MFF_LOG_CT_ZONE, 0, 32, ofpacts_p); + put_load(zone_ids->ct, MFF_LOG_CT_ZONE, 0, 16, ofpacts_p); } if (zone_ids->dnat) { put_load(zone_ids->dnat, MFF_LOG_DNAT_ZONE, 0, 32, ofpacts_p); @@ -1014,9 +1030,23 @@ put_local_common_flows(uint32_t dp_key, } } +static size_t +encap_ip_to_id(size_t n_encap_ips, const char **encap_ips, + const char *ip) +{ + for (size_t i = 0; i < n_encap_ips; i++) { + if (!strcmp(ip, encap_ips[i])) { + return i; + } + } + return 0; +} + static void load_logical_ingress_metadata(const struct sbrec_port_binding *binding, const struct zone_ids *zone_ids, + size_t n_encap_ips, + const char **encap_ips, struct ofpbuf *ofpacts_p) { put_zones_ofpacts(zone_ids, ofpacts_p); @@ -1026,6 +1056,14 @@ load_logical_ingress_metadata(const struct sbrec_port_binding *binding, uint32_t port_key = binding->tunnel_key; put_load(dp_key, MFF_LOG_DATAPATH, 0, 64, ofpacts_p); put_load(port_key, MFF_LOG_INPORT, 0, 32, ofpacts_p); + + /* Default encap_id 0. */ + size_t encap_id = 0; + if (encap_ips && binding->encap) { + encap_id = encap_ip_to_id(n_encap_ips, encap_ips, + binding->encap->ip); + } + put_load(encap_id, MFF_LOG_ENCAP_ID, 16, 16, ofpacts_p); } static const struct sbrec_port_binding * @@ -1070,7 +1108,7 @@ setup_rarp_activation_strategy(const struct sbrec_port_binding *binding, match_set_dl_type(&match, htons(ETH_TYPE_RARP)); match_set_in_port(&match, ofport); - load_logical_ingress_metadata(binding, zone_ids, &ofpacts); + load_logical_ingress_metadata(binding, zone_ids, 0, NULL, &ofpacts); encode_controller_op(ACTION_OPCODE_ACTIVATION_STRATEGY_RARP, NX_CTLR_NO_METER, &ofpacts); @@ -1384,7 +1422,7 @@ enforce_tunneling_for_multichassis_ports( } struct ovs_list *tuns = get_remote_tunnels(binding, chassis, - chassis_tunnels); + chassis_tunnels, NULL); if (ovs_list_is_empty(tuns)) { free(tuns); return; @@ -1446,6 +1484,8 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name, const struct sbrec_chassis *chassis, const struct physical_debug *debug, const struct if_status_mgr *if_mgr, + size_t n_encap_ips, + const char **encap_ips, struct ovn_desired_flow_table *flow_table, struct ofpbuf *ofpacts_p) { @@ -1479,9 +1519,10 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name, ofpact_put_CT_CLEAR(ofpacts_p); put_load(0, MFF_LOG_DNAT_ZONE, 0, 32, ofpacts_p); put_load(0, MFF_LOG_SNAT_ZONE, 0, 32, ofpacts_p); - put_load(0, MFF_LOG_CT_ZONE, 0, 32, ofpacts_p); + put_load(0, MFF_LOG_CT_ZONE, 0, 16, ofpacts_p); struct zone_ids peer_zones = get_zone_ids(peer, ct_zones); - load_logical_ingress_metadata(peer, &peer_zones, ofpacts_p); + load_logical_ingress_metadata(peer, &peer_zones, n_encap_ips, + encap_ips, ofpacts_p); put_load(0, MFF_LOG_FLAGS, 0, 32, ofpacts_p); put_load(0, MFF_LOG_OUTPORT, 0, 32, ofpacts_p); for (int i = 0; i < MFF_N_LOG_REGS; i++) { @@ -1697,7 +1738,8 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name, * as we're going to remove this with ofpbuf_pull() later. */ uint32_t ofpacts_orig_size = ofpacts_p->size; - load_logical_ingress_metadata(binding, &zone_ids, ofpacts_p); + load_logical_ingress_metadata(binding, &zone_ids, n_encap_ips, + encap_ips, ofpacts_p); if (!strcmp(binding->type, "localport")) { /* mark the packet as incoming from a localport */ @@ -1904,7 +1946,7 @@ consider_port_binding(struct ovsdb_idl_index *sbrec_port_binding_by_name, } else { put_remote_port_redirect_overlay( binding, mff_ovn_geneve, port_key, &match, ofpacts_p, - chassis, chassis_tunnels, flow_table); + chassis, chassis_tunnels, n_encap_ips, encap_ips, flow_table); } out: if (ha_ch_ordered) { @@ -2102,7 +2144,7 @@ consider_mc_group(struct ovsdb_idl_index *sbrec_port_binding_by_name, int zone_id = simap_get(ct_zones, port->logical_port); if (zone_id) { - put_load(zone_id, MFF_LOG_CT_ZONE, 0, 32, &ofpacts); + put_load(zone_id, MFF_LOG_CT_ZONE, 0, 16, &ofpacts); } const char *lport_name = (port->parent_port && *port->parent_port) ? @@ -2278,7 +2320,10 @@ physical_eval_port_binding(struct physical_ctx *p_ctx, p_ctx->patch_ofports, p_ctx->chassis_tunnels, pb, p_ctx->chassis, &p_ctx->debug, - p_ctx->if_mgr, flow_table, &ofpacts); + p_ctx->if_mgr, + p_ctx->n_encap_ips, + p_ctx->encap_ips, + flow_table, &ofpacts); ofpbuf_uninit(&ofpacts); } @@ -2402,7 +2447,10 @@ physical_run(struct physical_ctx *p_ctx, p_ctx->patch_ofports, p_ctx->chassis_tunnels, binding, p_ctx->chassis, &p_ctx->debug, - p_ctx->if_mgr, flow_table, &ofpacts); + p_ctx->if_mgr, + p_ctx->n_encap_ips, + p_ctx->encap_ips, + flow_table, &ofpacts); } /* Handle output to multicast groups, in tables 40 and 41. */ diff --git a/controller/physical.h b/controller/physical.h index 1f1ed55efa16..7fe8ee3c18ed 100644 --- a/controller/physical.h +++ b/controller/physical.h @@ -66,6 +66,8 @@ struct physical_ctx { struct shash *local_bindings; struct simap *patch_ofports; struct hmap *chassis_tunnels; + size_t n_encap_ips; + const char **encap_ips; struct physical_debug debug; }; diff --git a/include/ovn/logical-fields.h b/include/ovn/logical-fields.h index 272277ec4fa0..d3455a462a0d 100644 --- a/include/ovn/logical-fields.h +++ b/include/ovn/logical-fields.h @@ -37,7 +37,9 @@ enum ovn_controller_event { #define MFF_LOG_SNAT_ZONE MFF_REG12 /* conntrack snat zone for gateway router * (32 bits). */ #define MFF_LOG_CT_ZONE MFF_REG13 /* Logical conntrack zone for lports - * (32 bits). */ + * (0..15 of the 32 bits). */ +#define MFF_LOG_ENCAP_ID MFF_REG13 /* Encap ID for lports + * (16..31 of the 32 bits). */ #define MFF_LOG_INPORT MFF_REG14 /* Logical input port (32 bits). */ #define MFF_LOG_OUTPORT MFF_REG15 /* Logical output port (32 bits). */ diff --git a/ovn-architecture.7.xml b/ovn-architecture.7.xml index 96294fe2b795..bfd8680cedfc 100644 --- a/ovn-architecture.7.xml +++ b/ovn-architecture.7.xml @@ -1247,8 +1247,8 @@ chassis. This is initialized to 0 at the beginning of the logical - ingress pipeline. OVN stores this in Open vSwitch extension register - number 13. + ingress pipeline. OVN stores this in the lower 16 bits of the Open + vSwitch extension register number 13.
conntrack zone fields for routers
@@ -1263,6 +1263,20 @@ traffic (for SNATing) in Open vSwitch extension register number 12. +
Encap ID for logical ports
+
+ A field that records an ID that indicates which encapsulation IP should + be used when sending packets to a remote chassis, according to the + original input logical port. This is useful when there are multiple IPs + available for encapsulation. The value only has local significance and is + not meaningful between chassis. This is initialized to 0 at the beginning + of the logical + + ingress pipeline. OVN stores this in the higher 16 bits of the Open + vSwitch extension register number 13. +
+
logical flow flags
The logical flags are intended to handle keeping context between diff --git a/tests/ovn.at b/tests/ovn.at index 243fe0b8246c..e7f0c9681f60 100644 --- a/tests/ovn.at +++ b/tests/ovn.at @@ -30335,19 +30335,33 @@ AT_CLEANUP OVN_FOR_EACH_NORTHD([ -AT_SETUP([multiple encap ips tunnel creation]) +AT_SETUP([multiple encap ips selection based on VIF's encap_ip]) +AT_SKIP_IF([test $HAVE_SCAPY = no]) ovn_start net_add n1 +ovn-nbctl ls-add ls1 + # 2 HVs, each with 2 encap-ips. +# lspXY is the Yth LSP on hvX, with encap-ip 192.168.0.XY. for i in 1 2; do sim_add hv$i as hv$i ovs-vsctl add-br br-phys-$j ovn_attach n1 br-phys-$j 192.168.0.${i}1 ovs-vsctl set open . external_ids:ovn-encap-ip=192.168.0.${i}1,192.168.0.${i}2 + + for j in 1 2; do + ovs-vsctl add-port br-int vif$i$j -- set Interface vif$i$j \ + external_ids:iface-id=lsp$i$j \ + external_ids:encap-ip=192.168.0.$i$j \ + options:tx_pcap=hv$i/vif$i$j-tx.pcap options:rxq_pcap=hv$i/vif$i$j-rx.pcap + ovn-nbctl lsp-add ls1 lsp$i$j -- lsp-set-addresses lsp$i$j "f0:00:00:00:00:$i$j 10.0.0.$i$j" + + done done +wait_for_ports_up check ovn-nbctl --wait=hv sync check_tunnel_port() { @@ -30376,6 +30390,41 @@ check_tunnel_port hv2 br-int hv1@192.168.0.12%192.168.0.21 check_tunnel_port hv2 br-int hv1@192.168.0.11%192.168.0.22 check_tunnel_port hv2 br-int hv1@192.168.0.12%192.168.0.22 +vif_to_hv() { + case $1 in dnl ( + vif[[1]]?) echo hv1 ;; dnl ( + vif[[2]]?) echo hv2 ;; dnl ( + *) AT_FAIL_IF([:]) ;; + esac +} + +# check_packet_tunnel SRC_VIF DST_VIF +# Verify that a packet from SRC_VIF to DST_VIF goes through the corresponding +# tunnel that matches the VIFs' encap_ip configurations. +check_packet_tunnel() { + local src=$1 dst=$2 + local src_mac=f0:00:00:00:00:$src + local dst_mac=f0:00:00:00:00:$dst + local src_ip=10.0.0.$src + local dst_ip=10.0.0.$dst + local local_encap_ip=192.168.0.$src + local remote_encap_ip=192.168.0.$dst + local packet=$(fmt_pkt "Ether(dst='${dst_mac}', src='${src_mac}')/ \ + IP(dst='${dst_ip}', src='${src_ip}')/ \ + ICMP(type=8)") + hv=`vif_to_hv vif$src` + as $hv + echo "vif$src -> vif$dst should go through tunnel $local_encap_ip -> $remote_encap_ip" + tunnel_ofport=$(ovs-vsctl --bare --column=ofport find interface options:local_ip=$local_encap_ip options:remote_ip=$remote_encap_ip) + AT_CHECK([test $(ovs-appctl ofproto/trace br-int in_port=vif$src $packet | grep "output\:" | awk -F ':' '{ print $2 }') == $tunnel_ofport]) +} + +for i in 1 2; do + for j in 1 2; do + check_packet_tunnel 1$i 2$j + done +done + OVN_CLEANUP([hv1],[hv2]) AT_CLEANUP ])