From patchwork Tue Oct 5 20:24:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladislav Odintsov X-Patchwork-Id: 1536914 X-Patchwork-Delegate: zhouhan@gmail.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=UON4mTKE; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HP8Hm0nD1z9sPB for ; Wed, 6 Oct 2021 07:26:28 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 595F883E4C; Tue, 5 Oct 2021 20:26:26 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TPH5rP06VKUl; Tue, 5 Oct 2021 20:26:23 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id 0878F83DBB; Tue, 5 Oct 2021 20:26:22 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id C7E0AC0011; Tue, 5 Oct 2021 20:26:22 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 433D7C000D for ; Tue, 5 Oct 2021 20:26:21 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id E9A7583E39 for ; Tue, 5 Oct 2021 20:26:12 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Q_dmed6xPq1F for ; Tue, 5 Oct 2021 20:26:10 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com [IPv6:2a00:1450:4864:20::12b]) by smtp1.osuosl.org (Postfix) with ESMTPS id 2C97183E59 for ; Tue, 5 Oct 2021 20:26:10 +0000 (UTC) Received: by mail-lf1-x12b.google.com with SMTP id x27so921531lfa.9 for ; Tue, 05 Oct 2021 13:26:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=x+2Nye18M4Xhj3Yol1RfOA/JMfOFpNC/kevFuKznpXA=; b=UON4mTKEJLwzFGVBGOcDCO/aQ6uZCLWSiVPd4a6T9wfG7XmEF/6V6n2pThlY5NzCWY IkyFaX4KKKKBr9GjIXkeZrU5+Oi/S/8LOgqSYikDrC8661lSvCTQCKiSR6GR1EGnsk3q Zo+9L55abnXkCvdAqnGI8c5q9iup2rLNe4gfeeofiqH5aBbdnx9tt5VjOKg2lKhZYFzb PSM3PUfQNw39v479RJyr1FIaCLC+Z0i/7knkwq/CxJdh2rsMHhrIGmDE5Odllqpq6yQ7 Gm0cs0rCzwujdkYhAFDjRJ1qkNHt4xboemov3Pt8DrQKZfYJz9AZj0he3I6oguxIV7Ag jdjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=x+2Nye18M4Xhj3Yol1RfOA/JMfOFpNC/kevFuKznpXA=; b=7HJVc1g7CBDCnNFpeovvM9x9hAZEELzJTcl/TNaB1jiApCttJNayVbkuyu5ZdOBiA2 +3eY9xhMFIc1dNMAfsWcT48B+pQtdld/OsW6/vENIaHphXmVKl4Vr1rtqX2sU2murg37 FJlF935meFvSw49nsSa9hle27t66owx4NLpJ2/1mEoUoSbsA4i9HVYE2iqVEa5qwyhGg IxNBF39or9ff4EI3yoF5VHHXjTG1Dp8zfTHcFATmlDf21rvJq1qSVeUzLegBo5s35I21 1pLICWKHBqmCcvZpEMiOIejVL84cQ0SrVo+ZHeNdXVl3UWH+M97Xt0Z0SsVVo2U5L2Rr bA/g== X-Gm-Message-State: AOAM531FlNb1VrxmAeIZNTZTmqYcFXkpB3VQz5i/El9CXVLYJ8Q0PkyW 5vbQ8d0V04w9iTiU57JVgnr80AODz3PV5w== X-Google-Smtp-Source: ABdhPJxwU5F59I6egn4ghOKooLUhS3T6Itb96UsztPjUzrPSNd4lhMJW6yI63c+SqUEtiOnwQDWvQw== X-Received: by 2002:a2e:80cd:: with SMTP id r13mr23719659ljg.415.1633465567104; Tue, 05 Oct 2021 13:26:07 -0700 (PDT) Received: from localhost.localdomain (109-252-131-59.dynamic.spd-mgts.ru. [109.252.131.59]) by smtp.gmail.com with ESMTPSA id 12sm2058870lfz.259.2021.10.05.13.26.06 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 05 Oct 2021 13:26:06 -0700 (PDT) From: Vladislav Odintsov To: dev@openvswitch.org Date: Tue, 5 Oct 2021 23:24:41 +0300 Message-Id: <20211005202442.85322-4-odivlad@gmail.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20211005202442.85322-1-odivlad@gmail.com> References: <20211005202442.85322-1-odivlad@gmail.com> MIME-Version: 1.0 Cc: Vladislav Odintsov Subject: [ovs-dev] [PATCH ovn v6 3/4] ic: add support for routing tables in adv/learn routes X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Previously support for multiple routing tables was added to northd code. This commit expands support for multiple routing tables by adding support of advertising and learning routes with their routing table information. To utilize such feature, user must: 1. create Logical Router in each AZ; 2. create IC transit switch for each routing table, that he/she needs; 3. connect each TS with this LR; 4. assign routing table for TS's LRP (ovn-nbctl lrp-set-options route_table=<>); 5. enable routes sync (turn on learning and advertising routes in NB_Global table); 6. create LRPs for subnets in LR, create static routes with supplying route_table parameter. Note 1: routes for directly-connected networks will be learned to global routing table and if Logical Routers have more than one Transit Switch, which interconnects them, directly-connected routes will be added via each transit switch port and configured as ECMP routes. Note 2: static routes within route tables will be advertised and learned only if interconnecting transit switch's LRPs will have options:route_table same value as route's route_table value. Signed-off-by: Vladislav Odintsov Reviewed-by: Numan Siddique --- NEWS | 4 + ic/ovn-ic.c | 540 ++++++++++++++++++++++++++++---------------- ovn-ic-sb.ovsschema | 5 +- ovn-ic-sb.xml | 18 ++ tests/ovn-ic.at | 440 ++++++++++++++++++++++++++++++++++++ 5 files changed, 811 insertions(+), 196 deletions(-) diff --git a/NEWS b/NEWS index 5e93f813a..dcb70f944 100644 --- a/NEWS +++ b/NEWS @@ -16,6 +16,10 @@ OVN v21.09.0 - xx xxx xxxx - Allow static routes without nexthops. - Enabled logical dp groups as a default. CMS should disable it if not desired. + - Added support for multiple routing tables in Logical Router Static Routes + and LRPs. OVN Interconnection supports routes' route tables as well. + This requires to update schemas for OVN_Northdbound and OVN_IC_Southbound + DBs. OVN v21.06.0 - 18 Jun 2021 ------------------------- diff --git a/ic/ovn-ic.c b/ic/ovn-ic.c index 303e93a4f..02bd28b2b 100644 --- a/ic/ovn-ic.c +++ b/ic/ovn-ic.c @@ -63,9 +63,11 @@ struct ic_context { struct ovsdb_idl_txn *ovninb_txn; struct ovsdb_idl_txn *ovnisb_txn; struct ovsdb_idl_index *nbrec_ls_by_name; + struct ovsdb_idl_index *nbrec_lrp_by_name; struct ovsdb_idl_index *nbrec_port_by_name; struct ovsdb_idl_index *sbrec_chassis_by_name; struct ovsdb_idl_index *sbrec_port_binding_by_name; + struct ovsdb_idl_index *icnbrec_transit_switch_by_name; struct ovsdb_idl_index *icsbrec_port_binding_by_az; struct ovsdb_idl_index *icsbrec_port_binding_by_ts; struct ovsdb_idl_index *icsbrec_port_binding_by_ts_az; @@ -773,7 +775,7 @@ port_binding_run(struct ic_context *ctx, icsbrec_port_binding_index_set_transit_switch(isb_pb_key, ts->name); ICSBREC_PORT_BINDING_FOR_EACH_EQUAL (isb_pb, isb_pb_key, - ctx->icsbrec_port_binding_by_ts) { + ctx->icsbrec_port_binding_by_ts) { if (isb_pb->availability_zone == az) { shash_add(&local_pbs, isb_pb->logical_port, isb_pb); shash_find_and_delete(&isb_all_local_pbs, @@ -844,7 +846,9 @@ port_binding_run(struct ic_context *ctx, struct ic_router_info { struct hmap_node node; const struct nbrec_logical_router *lr; /* key of hmap */ - const struct icsbrec_port_binding *isb_pb; + const struct icsbrec_port_binding **isb_pbs; + size_t n_isb_pbs; + size_t n_allocated_isb_pbs; struct hmap routes_learned; }; @@ -854,6 +858,7 @@ struct ic_route_info { struct in6_addr prefix; unsigned int plen; struct in6_addr nexthop; + const char *route_table; /* Either nb_route or nb_lrp is set and the other one must be NULL. * - For a route that is learned from IC-SB, or a static route that is @@ -875,13 +880,15 @@ ic_route_hash(const struct in6_addr *prefix, unsigned int plen, static struct ic_route_info * ic_route_find(struct hmap *routes, const struct in6_addr *prefix, - unsigned int plen, const struct in6_addr *nexthop) + unsigned int plen, const struct in6_addr *nexthop, + char *route_table) { struct ic_route_info *r; uint32_t hash = ic_route_hash(prefix, plen, nexthop); HMAP_FOR_EACH_WITH_HASH (r, node, hash, routes) { if (ipv6_addr_equals(&r->prefix, prefix) && r->plen == plen && + !strcmp(r->route_table ? r->route_table : "", route_table) && ipv6_addr_equals(&r->nexthop, nexthop)) { return r; } @@ -926,11 +933,19 @@ add_to_routes_learned(struct hmap *routes_learned, &prefix, &plen, &nexthop)) { return false; } + + if (ic_route_find(routes_learned, &prefix, plen, &nexthop, + nb_route->route_table)) { + /* Route is already added to learned in previous iteration. */ + return true; + } + struct ic_route_info *ic_route = xzalloc(sizeof *ic_route); ic_route->prefix = prefix; ic_route->plen = plen; ic_route->nexthop = nexthop; ic_route->nb_route = nb_route; + ic_route->route_table = nb_route->route_table; hmap_insert(routes_learned, &ic_route->node, ic_route_hash(&prefix, plen, &nexthop)); return true; @@ -1069,8 +1084,17 @@ static void add_to_routes_ad(struct hmap *routes_ad, const struct nbrec_logical_router_static_route *nb_route, const struct lport_addresses *nexthop_addresses, - const struct smap *nb_options) + const struct smap *nb_options, const char *route_table) { + if (strcmp(route_table, nb_route->route_table)) { + if (VLOG_IS_DBG_ENABLED()) { + VLOG_DBG("Skip advertising route %s -> %s as its route table %s !=" + " %s of TS port", nb_route->ip_prefix, nb_route->nexthop, + nb_route->route_table, route_table); + } + return; + } + struct in6_addr prefix, nexthop; unsigned int plen; if (!parse_route(nb_route->ip_prefix, nb_route->nexthop, @@ -1088,11 +1112,33 @@ add_to_routes_ad(struct hmap *routes_ad, return; } + if (VLOG_IS_DBG_ENABLED()) { + struct ds msg = DS_EMPTY_INITIALIZER; + + ds_put_format(&msg, "Advertising static route: %s -> %s, ic nexthop: ", + nb_route->ip_prefix, nb_route->nexthop); + + if (IN6_IS_ADDR_V4MAPPED(&nexthop)) { + ds_put_format(&msg, IP_FMT, + IP_ARGS(in6_addr_get_mapped_ipv4(&nexthop))); + } else { + ipv6_format_addr(&nexthop, &msg); + } + + ds_put_format(&msg, ", route_table: %s", strlen(nb_route->route_table) + ? nb_route->route_table + : "global"); + + VLOG_DBG("%s", ds_cstr(&msg)); + ds_destroy(&msg); + } + struct ic_route_info *ic_route = xzalloc(sizeof *ic_route); ic_route->prefix = prefix; ic_route->plen = plen; ic_route->nexthop = nexthop; ic_route->nb_route = nb_route; + ic_route->route_table = nb_route->route_table; hmap_insert(routes_ad, &ic_route->node, ic_route_hash(&prefix, plen, &nexthop)); } @@ -1124,8 +1170,8 @@ add_network_to_routes_ad(struct hmap *routes_ad, const char *network, if (VLOG_IS_DBG_ENABLED()) { struct ds msg = DS_EMPTY_INITIALIZER; - ds_put_format(&msg, "Route ad: direct network %s of lrp %s, nexthop ", - network, nb_lrp->name); + ds_put_format(&msg, "Adding direct network route to global routing " + "table: %s of lrp %s, nexthop ", network, nb_lrp->name); if (IN6_IS_ADDR_V4MAPPED(&nexthop)) { ds_put_format(&msg, IP_FMT, @@ -1143,13 +1189,15 @@ add_network_to_routes_ad(struct hmap *routes_ad, const char *network, ic_route->plen = plen; ic_route->nexthop = nexthop; ic_route->nb_lrp = nb_lrp; + + /* directly-connected routes go to global route table */ + ic_route->route_table = NULL; hmap_insert(routes_ad, &ic_route->node, ic_route_hash(&prefix, plen, &nexthop)); } static bool -route_need_learn(struct in6_addr *prefix, - unsigned int plen, +route_need_learn(struct in6_addr *prefix, unsigned int plen, const struct smap *nb_options) { if (!smap_get_bool(nb_options, "ic-route-learn", false)) { @@ -1172,73 +1220,149 @@ route_need_learn(struct in6_addr *prefix, return true; } +static const char * +get_lrp_name_by_ts_port_name(struct ic_context *ctx, const char *ts_port_name) +{ + const struct nbrec_logical_switch_port *nb_lsp; + const struct nbrec_logical_switch_port *nb_lsp_key = + nbrec_logical_switch_port_index_init_row(ctx->nbrec_port_by_name); + nbrec_logical_switch_port_index_set_name(nb_lsp_key, ts_port_name); + nb_lsp = nbrec_logical_switch_port_index_find(ctx->nbrec_port_by_name, + nb_lsp_key); + nbrec_logical_switch_port_index_destroy_row(nb_lsp_key); + + if (!nb_lsp) { + return NULL; + } + + return smap_get(&nb_lsp->options, "router-port"); +} + +static const char * +get_route_table_by_lrp_name(struct ic_context *ctx, const char *lrp_name) +{ + const struct nbrec_logical_router_port *lrp; + const struct nbrec_logical_router_port *lrp_key = + nbrec_logical_router_port_index_init_row(ctx->nbrec_lrp_by_name); + nbrec_logical_router_port_index_set_name(lrp_key, lrp_name); + lrp = nbrec_logical_router_port_index_find(ctx->nbrec_lrp_by_name, + lrp_key); + nbrec_logical_router_port_index_destroy_row(lrp_key); + + if (lrp) { + return smap_get_def(&lrp->options, "route_table", ""); + } + return ""; /* Global route table */ +} + +static bool +lrp_is_ts_port(struct ic_context *ctx, struct ic_router_info *ic_lr, + const char *lrp_name) +{ + const struct icsbrec_port_binding *isb_pb; + const char *ts_lrp_name; + for (int i = 0; i < ic_lr->n_isb_pbs; i++) { + isb_pb = ic_lr->isb_pbs[i]; + ts_lrp_name = get_lrp_name_by_ts_port_name(ctx, isb_pb->logical_port); + if (!strcmp(ts_lrp_name, lrp_name)) { + return true; + } + } + return false; +} + static void -sync_learned_route(struct ic_context *ctx, - const struct icsbrec_availability_zone *az, - struct ic_router_info *ic_lr) +sync_learned_routes(struct ic_context *ctx, + const struct icsbrec_availability_zone *az, + struct ic_router_info *ic_lr) { ovs_assert(ctx->ovnnb_txn); - const struct icsbrec_route *isb_route; - const struct icsbrec_route *isb_route_key = - icsbrec_route_index_init_row(ctx->icsbrec_route_by_ts); + const struct icsbrec_route *isb_route, *isb_route_key; - icsbrec_route_index_set_transit_switch(isb_route_key, - ic_lr->isb_pb->transit_switch); + const struct nbrec_nb_global *nb_global = + nbrec_nb_global_first(ctx->ovnnb_idl); + ovs_assert(nb_global); - ICSBREC_ROUTE_FOR_EACH_EQUAL (isb_route, isb_route_key, - ctx->icsbrec_route_by_ts) { - if (isb_route->availability_zone == az) { - continue; - } - struct in6_addr prefix, nexthop; - unsigned int plen; - if (!parse_route(isb_route->ip_prefix, isb_route->nexthop, - &prefix, &plen, &nexthop)) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_WARN_RL(&rl, "Bad route format in IC-SB: %s -> %s. Ignored.", - isb_route->ip_prefix, isb_route->nexthop); - continue; - } - const struct nbrec_nb_global *nb_global = - nbrec_nb_global_first(ctx->ovnnb_idl); - ovs_assert(nb_global); - if (!route_need_learn(&prefix, plen, &nb_global->options)) { - continue; - } - struct ic_route_info *route_learned - = ic_route_find(&ic_lr->routes_learned, &prefix, plen, &nexthop); - if (route_learned) { - /* Sync external-ids */ - struct uuid ext_id; - smap_get_uuid(&route_learned->nb_route->external_ids, - "ic-learned-route", &ext_id); - if (!uuid_equals(&ext_id, &isb_route->header_.uuid)) { + const char *lrp_name, *ts_route_table; + const struct icsbrec_port_binding *isb_pb; + for (int i = 0; i < ic_lr->n_isb_pbs; i++) { + isb_pb = ic_lr->isb_pbs[i]; + lrp_name = get_lrp_name_by_ts_port_name(ctx, isb_pb->logical_port); + ts_route_table = get_route_table_by_lrp_name(ctx, lrp_name); + + isb_route_key = icsbrec_route_index_init_row(ctx->icsbrec_route_by_ts); + icsbrec_route_index_set_transit_switch(isb_route_key, + isb_pb->transit_switch); + + ICSBREC_ROUTE_FOR_EACH_EQUAL (isb_route, isb_route_key, + ctx->icsbrec_route_by_ts) { + if (isb_route->availability_zone == az) { + continue; + } + + if (strlen(isb_route->route_table) && + strcmp(isb_route->route_table, ts_route_table)) { + if (VLOG_IS_DBG_ENABLED()) { + VLOG_DBG("Skip learning static route %s -> %s as either " + "its route table %s != %s of TS port or ", + isb_route->ip_prefix, isb_route->nexthop, + isb_route->route_table, ts_route_table); + } + continue; + } + + struct in6_addr prefix, nexthop; + unsigned int plen; + if (!parse_route(isb_route->ip_prefix, isb_route->nexthop, + &prefix, &plen, &nexthop)) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "Bad route format in IC-SB: %s -> %s. " + "Ignored.", isb_route->ip_prefix, + isb_route->nexthop); + continue; + } + if (!route_need_learn(&prefix, plen, &nb_global->options)) { + continue; + } + struct ic_route_info *route_learned + = ic_route_find(&ic_lr->routes_learned, &prefix, plen, + &nexthop, isb_route->route_table); + if (route_learned) { + /* Sync external-ids */ + struct uuid ext_id; + smap_get_uuid(&route_learned->nb_route->external_ids, + "ic-learned-route", &ext_id); + if (!uuid_equals(&ext_id, &isb_route->header_.uuid)) { + char *uuid_s = + xasprintf(UUID_FMT, + UUID_ARGS(&isb_route->header_.uuid)); + nbrec_logical_router_static_route_update_external_ids_setkey( + route_learned->nb_route, "ic-learned-route", uuid_s); + free(uuid_s); + } + hmap_remove(&ic_lr->routes_learned, &route_learned->node); + free(route_learned); + } else { + /* Create the missing route in NB. */ + const struct nbrec_logical_router_static_route *nb_route = + nbrec_logical_router_static_route_insert(ctx->ovnnb_txn); + nbrec_logical_router_static_route_set_ip_prefix(nb_route, + isb_route->ip_prefix); + nbrec_logical_router_static_route_set_nexthop(nb_route, + isb_route->nexthop); char *uuid_s = xasprintf(UUID_FMT, UUID_ARGS(&isb_route->header_.uuid)); + nbrec_logical_router_static_route_set_route_table(nb_route, + isb_route->route_table); nbrec_logical_router_static_route_update_external_ids_setkey( - route_learned->nb_route, "ic-learned-route", uuid_s); + nb_route, "ic-learned-route", uuid_s); free(uuid_s); + nbrec_logical_router_update_static_routes_addvalue(ic_lr->lr, + nb_route); } - hmap_remove(&ic_lr->routes_learned, &route_learned->node); - free(route_learned); - } else { - /* Create the missing route in NB. */ - const struct nbrec_logical_router_static_route *nb_route = - nbrec_logical_router_static_route_insert(ctx->ovnnb_txn); - nbrec_logical_router_static_route_set_ip_prefix( - nb_route, isb_route->ip_prefix); - nbrec_logical_router_static_route_set_nexthop( - nb_route, isb_route->nexthop); - char *uuid_s = xasprintf(UUID_FMT, - UUID_ARGS(&isb_route->header_.uuid)); - nbrec_logical_router_static_route_update_external_ids_setkey( - nb_route, "ic-learned-route", uuid_s); - free(uuid_s); - nbrec_logical_router_update_static_routes_addvalue( - ic_lr->lr, nb_route); } + icsbrec_route_index_destroy_row(isb_route_key); } - icsbrec_route_index_destroy_row(isb_route_key); /* Delete extra learned routes. */ struct ic_route_info *route_learned, *next; @@ -1271,10 +1395,10 @@ ad_route_sync_external_ids(const struct ic_route_info *route_adv, /* Sync routes from routes_ad to IC-SB. */ static void -advertise_route(struct ic_context *ctx, - const struct icsbrec_availability_zone *az, - const char *ts_name, - struct hmap *routes_ad) +advertise_routes(struct ic_context *ctx, + const struct icsbrec_availability_zone *az, + const char *ts_name, + struct hmap *routes_ad) { ovs_assert(ctx->ovnisb_txn); const struct icsbrec_route *isb_route; @@ -1298,7 +1422,8 @@ advertise_route(struct ic_context *ctx, continue; } struct ic_route_info *route_adv = - ic_route_find(routes_ad, &prefix, plen, &nexthop); + ic_route_find(routes_ad, &prefix, plen, &nexthop, + isb_route->route_table); if (!route_adv) { /* Delete the extra route from IC-SB. */ VLOG_DBG("Delete route %s -> %s from IC-SB, which is not found" @@ -1338,6 +1463,9 @@ advertise_route(struct ic_context *ctx, } icsbrec_route_set_ip_prefix(isb_route, prefix_s); icsbrec_route_set_nexthop(isb_route, nexthop_s); + icsbrec_route_set_route_table(isb_route, route_adv->route_table + ? route_adv->route_table + : ""); free(prefix_s); free(nexthop_s); @@ -1348,23 +1476,97 @@ advertise_route(struct ic_context *ctx, } } -static const char * -get_lrp_name_by_ts_port_name(struct ic_context *ctx, - const char *ts_port_name) +static void +build_ts_routes_to_adv(struct ic_context *ctx, + struct ic_router_info *ic_lr, + struct hmap *routes_ad, + struct lport_addresses *ts_port_addrs, + const struct nbrec_nb_global *nb_global, + const char *ts_route_table) { - const struct nbrec_logical_switch_port *nb_lsp; - const struct nbrec_logical_switch_port *nb_lsp_key = - nbrec_logical_switch_port_index_init_row(ctx->nbrec_port_by_name); - nbrec_logical_switch_port_index_set_name(nb_lsp_key, ts_port_name); - nb_lsp = nbrec_logical_switch_port_index_find(ctx->nbrec_port_by_name, - nb_lsp_key); - nbrec_logical_switch_port_index_destroy_row(nb_lsp_key); + const struct nbrec_logical_router *lr = ic_lr->lr; + + /* Check static routes of the LR */ + for (int i = 0; i < lr->n_static_routes; i++) { + const struct nbrec_logical_router_static_route *nb_route + = lr->static_routes[i]; + struct uuid isb_uuid; + if (smap_get_uuid(&nb_route->external_ids, "ic-learned-route", + &isb_uuid)) { + /* It is a learned route */ + if (!add_to_routes_learned(&ic_lr->routes_learned, nb_route)) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "Bad format of learned route in NB: " + "%s -> %s. Delete it.", nb_route->ip_prefix, + nb_route->nexthop); + nbrec_logical_router_update_static_routes_delvalue(lr, + nb_route); + } + } else { + /* It may be a route to be advertised */ + add_to_routes_ad(routes_ad, nb_route, ts_port_addrs, + &nb_global->options, ts_route_table); + } + } - if (!nb_lsp) { - return NULL; + /* Check directly-connected subnets of the LR */ + for (int i = 0; i < lr->n_ports; i++) { + const struct nbrec_logical_router_port *lrp = lr->ports[i]; + if (!lrp_is_ts_port(ctx, ic_lr, lrp->name)) { + for (int j = 0; j < lrp->n_networks; j++) { + add_network_to_routes_ad(routes_ad, lrp->networks[j], lrp, + ts_port_addrs, + &nb_global->options); + } + } else { + /* The router port of the TS port is ignored. */ + VLOG_DBG("Skip advertising direct route of lrp %s (TS port)", + lrp->name); + } } +} - return smap_get(&nb_lsp->options, "router-port"); +static void +advertise_lr_routes(struct ic_context *ctx, + const struct icsbrec_availability_zone *az, + struct ic_router_info *ic_lr) +{ + const struct nbrec_nb_global *nb_global = + nbrec_nb_global_first(ctx->ovnnb_idl); + ovs_assert(nb_global); + + const struct icsbrec_port_binding *isb_pb; + const char *lrp_name, *ts_name, *route_table; + struct lport_addresses ts_port_addrs; + const struct nbrec_logical_router *lr = ic_lr->lr; + const struct icnbrec_transit_switch *key; + + struct hmap routes_ad = HMAP_INITIALIZER(&routes_ad); + for (int i = 0; i < ic_lr->n_isb_pbs; i++) { + isb_pb = ic_lr->isb_pbs[i]; + key = icnbrec_transit_switch_index_init_row( + ctx->icnbrec_transit_switch_by_name); + icnbrec_transit_switch_index_set_name(key, isb_pb->transit_switch); + ts_name = icnbrec_transit_switch_index_find( + ctx->icnbrec_transit_switch_by_name, key)->name; + icnbrec_transit_switch_index_destroy_row(key); + + if (!extract_lsp_addresses(isb_pb->address, &ts_port_addrs)) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_INFO_RL(&rl, "Route sync ignores port %s on ts %s for router" + " %s because the addresses are invalid.", + isb_pb->logical_port, isb_pb->transit_switch, + lr->name); + continue; + } + lrp_name = get_lrp_name_by_ts_port_name(ctx, isb_pb->logical_port); + route_table = get_route_table_by_lrp_name(ctx, lrp_name); + build_ts_routes_to_adv(ctx, ic_lr, &routes_ad, &ts_port_addrs, + nb_global, route_table); + advertise_routes(ctx, az, ts_name, &routes_ad); + destroy_lport_addresses(&ts_port_addrs); + } + hmap_destroy(&routes_ad); } static void @@ -1375,130 +1577,70 @@ route_run(struct ic_context *ctx, return; } - const struct nbrec_nb_global *nb_global = - nbrec_nb_global_first(ctx->ovnnb_idl); - ovs_assert(nb_global); - - const struct icnbrec_transit_switch *ts; - ICNBREC_TRANSIT_SWITCH_FOR_EACH (ts, ctx->ovninb_idl) { - struct hmap ic_lrs = HMAP_INITIALIZER(&ic_lrs); - struct hmap routes_ad = HMAP_INITIALIZER(&routes_ad); - - const struct icsbrec_port_binding *isb_pb; - const struct icsbrec_port_binding *isb_pb_key = - icsbrec_port_binding_index_init_row( - ctx->icsbrec_port_binding_by_ts_az); - icsbrec_port_binding_index_set_transit_switch(isb_pb_key, ts->name); - icsbrec_port_binding_index_set_availability_zone(isb_pb_key, az); - - /* Each port on TS maps to a logical router, which is stored in the - * external_ids:router-id of the IC SB port_binding record. */ - ICSBREC_PORT_BINDING_FOR_EACH_EQUAL (isb_pb, isb_pb_key, - ctx->icsbrec_port_binding_by_ts_az) { - const char *ts_lrp_name = - get_lrp_name_by_ts_port_name(ctx, isb_pb->logical_port); - if (!ts_lrp_name) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_WARN_RL(&rl, "Route sync ignores port %s on ts %s " - "because logical router port is not found in NB.", - isb_pb->logical_port, ts->name); - continue; - } + struct hmap ic_lrs = HMAP_INITIALIZER(&ic_lrs); + const struct icsbrec_port_binding *isb_pb; + const struct icsbrec_port_binding *isb_pb_key = + icsbrec_port_binding_index_init_row(ctx->icsbrec_port_binding_by_az); + icsbrec_port_binding_index_set_availability_zone(isb_pb_key, az); - struct uuid lr_uuid; - if (!smap_get_uuid(&isb_pb->external_ids, "router-id", &lr_uuid)) { - VLOG_DBG("IC-SB Port_Binding %s doesn't have " - "external_ids:router-id set.", isb_pb->logical_port); - continue; - } - const struct nbrec_logical_router *lr - = nbrec_logical_router_get_for_uuid(ctx->ovnnb_idl, &lr_uuid); - if (!lr) { - continue; - } + /* Each port on TS maps to a logical router, which is stored in the + * external_ids:router-id of the IC SB port_binding record. + * Here we build info for interconnected Logical Router: + * collect IC Port Binding to process routes sync later on. */ + ICSBREC_PORT_BINDING_FOR_EACH_EQUAL (isb_pb, isb_pb_key, + ctx->icsbrec_port_binding_by_az) + { + const char *ts_lrp_name = + get_lrp_name_by_ts_port_name(ctx, isb_pb->logical_port); + if (!ts_lrp_name) { + static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); + VLOG_WARN_RL(&rl, "Route sync ignores port %s on ts %s because " + "logical router port is not found in NB. Deleting it", + isb_pb->logical_port, isb_pb->transit_switch); + icsbrec_port_binding_delete(isb_pb); + continue; + } - if (ic_router_find(&ic_lrs, lr)) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_INFO_RL(&rl, "Route sync ignores port %s on ts %s for " - "router %s because the router has another port " - "connected to same ts.", isb_pb->logical_port, - ts->name, lr->name); - continue; - } + struct uuid lr_uuid; + if (!smap_get_uuid(&isb_pb->external_ids, "router-id", &lr_uuid)) { + VLOG_DBG("IC-SB Port_Binding %s doesn't have " + "external_ids:router-id set.", isb_pb->logical_port); + continue; + } - struct lport_addresses ts_port_addrs; - if (!extract_lsp_addresses(isb_pb->address, &ts_port_addrs)) { - static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_INFO_RL(&rl, "Route sync ignores port %s on ts %s for " - "router %s because the addresses are invalid.", - isb_pb->logical_port, ts->name, lr->name); - continue; - } + const struct nbrec_logical_router *lr + = nbrec_logical_router_get_for_uuid(ctx->ovnnb_idl, &lr_uuid); + if (!lr) { + continue; + } - struct ic_router_info *ic_lr = xzalloc(sizeof *ic_lr); + struct ic_router_info *ic_lr = ic_router_find(&ic_lrs, lr); + if (!ic_lr) { + ic_lr = xzalloc(sizeof *ic_lr); ic_lr->lr = lr; - ic_lr->isb_pb = isb_pb; hmap_init(&ic_lr->routes_learned); hmap_insert(&ic_lrs, &ic_lr->node, uuid_hash(&lr->header_.uuid)); - - /* Check static routes of the LR */ - for (int i = 0; i < lr->n_static_routes; i++) { - const struct nbrec_logical_router_static_route *nb_route - = lr->static_routes[i]; - struct uuid isb_uuid; - if (smap_get_uuid(&nb_route->external_ids, - "ic-learned-route", &isb_uuid)) { - /* It is a learned route */ - if (!add_to_routes_learned(&ic_lr->routes_learned, - nb_route)) { - static struct vlog_rate_limit rl = - VLOG_RATE_LIMIT_INIT(5, 1); - VLOG_WARN_RL(&rl, "Bad format of learned route in NB:" - " %s -> %s. Delete it.", - nb_route->ip_prefix, nb_route->nexthop); - nbrec_logical_router_update_static_routes_delvalue( - lr, nb_route); - } - } else { - /* It may be a route to be advertised */ - add_to_routes_ad(&routes_ad, nb_route, &ts_port_addrs, - &nb_global->options); - } - } - - /* Check direct-connected subnets of the LR */ - for (int i = 0; i < lr->n_ports; i++) { - const struct nbrec_logical_router_port *lrp = lr->ports[i]; - if (!strcmp(lrp->name, ts_lrp_name)) { - /* The router port of the TS port is ignored. */ - VLOG_DBG("Route ad: skip lrp %s (TS port: %s)", - lrp->name, isb_pb->logical_port); - continue; - } - - for (int j = 0; j < lrp->n_networks; j++) { - add_network_to_routes_ad(&routes_ad, lrp->networks[j], - lrp, &ts_port_addrs, - &nb_global->options); - } - } - - destroy_lport_addresses(&ts_port_addrs); } - icsbrec_port_binding_index_destroy_row(isb_pb_key); - - advertise_route(ctx, az, ts->name, &routes_ad); - hmap_destroy(&routes_ad); - struct ic_router_info *ic_lr, *next; - HMAP_FOR_EACH_SAFE (ic_lr, next, node, &ic_lrs) { - sync_learned_route(ctx, az, ic_lr); - hmap_destroy(&ic_lr->routes_learned); - hmap_remove(&ic_lrs, &ic_lr->node); - free(ic_lr); + if (ic_lr->n_isb_pbs == ic_lr->n_allocated_isb_pbs) { + ic_lr->isb_pbs = x2nrealloc(ic_lr->isb_pbs, + &ic_lr->n_allocated_isb_pbs, + sizeof *ic_lr->isb_pbs); } - hmap_destroy(&ic_lrs); + ic_lr->isb_pbs[ic_lr->n_isb_pbs++] = isb_pb; } + icsbrec_port_binding_index_destroy_row(isb_pb_key); + + struct ic_router_info *ic_lr, *next; + HMAP_FOR_EACH_SAFE (ic_lr, next, node, &ic_lrs) { + advertise_lr_routes(ctx, az, ic_lr); + sync_learned_routes(ctx, az, ic_lr); + free(ic_lr->isb_pbs); + hmap_destroy(&ic_lr->routes_learned); + hmap_remove(&ic_lrs, &ic_lr->node); + free(ic_lr); + } + hmap_destroy(&ic_lrs); } static void @@ -1696,6 +1838,9 @@ main(int argc, char *argv[]) struct ovsdb_idl_index *nbrec_port_by_name = ovsdb_idl_index_create1(ovnnb_idl_loop.idl, &nbrec_logical_switch_port_col_name); + struct ovsdb_idl_index *nbrec_lrp_by_name + = ovsdb_idl_index_create1(ovnnb_idl_loop.idl, + &nbrec_logical_router_port_col_name); struct ovsdb_idl_index *sbrec_port_binding_by_name = ovsdb_idl_index_create1(ovnsb_idl_loop.idl, &sbrec_port_binding_col_logical_port); @@ -1703,6 +1848,10 @@ main(int argc, char *argv[]) = ovsdb_idl_index_create1(ovnsb_idl_loop.idl, &sbrec_chassis_col_name); + struct ovsdb_idl_index *icnbrec_transit_switch_by_name + = ovsdb_idl_index_create1(ovninb_idl_loop.idl, + &icnbrec_transit_switch_col_name); + struct ovsdb_idl_index *icsbrec_port_binding_by_az = ovsdb_idl_index_create1(ovnisb_idl_loop.idl, &icsbrec_port_binding_col_availability_zone); @@ -1761,9 +1910,12 @@ main(int argc, char *argv[]) .ovnisb_idl = ovnisb_idl_loop.idl, .ovnisb_txn = ovsdb_idl_loop_run(&ovnisb_idl_loop), .nbrec_ls_by_name = nbrec_ls_by_name, + .nbrec_lrp_by_name = nbrec_lrp_by_name, .nbrec_port_by_name = nbrec_port_by_name, .sbrec_port_binding_by_name = sbrec_port_binding_by_name, .sbrec_chassis_by_name = sbrec_chassis_by_name, + .icnbrec_transit_switch_by_name = + icnbrec_transit_switch_by_name, .icsbrec_port_binding_by_az = icsbrec_port_binding_by_az, .icsbrec_port_binding_by_ts = icsbrec_port_binding_by_ts, .icsbrec_port_binding_by_ts_az = icsbrec_port_binding_by_ts_az, diff --git a/ovn-ic-sb.ovsschema b/ovn-ic-sb.ovsschema index 5364b21b4..140ced149 100644 --- a/ovn-ic-sb.ovsschema +++ b/ovn-ic-sb.ovsschema @@ -1,7 +1,7 @@ { "name": "OVN_IC_Southbound", - "version": "1.0.0", - "cksum": "108951192 6585", + "version": "1.1.0", + "cksum": "3076915724 6636", "tables": { "IC_SB_Global": { "columns": { @@ -92,6 +92,7 @@ "transit_switch": {"type": "string"}, "availability_zone": {"type": {"key": {"type": "uuid", "refTable": "Availability_Zone"}}}, + "route_table": {"type": "string"}, "ip_prefix": {"type": "string"}, "nexthop": {"type": "string"}, "external_ids": { diff --git a/ovn-ic-sb.xml b/ovn-ic-sb.xml index 3582cff47..2f2ecf952 100644 --- a/ovn-ic-sb.xml +++ b/ovn-ic-sb.xml @@ -306,6 +306,24 @@ The availability zone that has advertised the route. + + Route table within which this route was created. + Empty value means "global" routing table. +

+ Routes for directly-connected networks will be + learned to global routing table and if Logical Routers + have more than one Transit Switch, which interconnects + them, directly-connected routes will be added via each + transit switch port and configured as ECMP routes. +

+

+ Static routes within route tables will be advertised + and learned only if interconnecting transit switch's LRPs + will have options:route_table same value as + route's route_table value. +

+ + IP prefix of this route (e.g. 192.168.100.0/24). diff --git a/tests/ovn-ic.at b/tests/ovn-ic.at index 3aab54362..5803f76e9 100644 --- a/tests/ovn-ic.at +++ b/tests/ovn-ic.at @@ -430,3 +430,443 @@ OVN_CLEANUP_IC([az1], [az2]) AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD([ +AT_SETUP([ovn-ic -- route sync -- route tables]) + +ovn_init_ic_db +ovn-ic-nbctl ts-add ts1 + +for i in 1 2; do + ovn_start az$i + ovn_as az$i + + # Enable route learning at AZ level + ovn-nbctl set nb_global . options:ic-route-learn=true + # Enable route advertising at AZ level + ovn-nbctl set nb_global . options:ic-route-adv=true + + # Create LRP and connect to TS + ovn-nbctl lr-add lr$i + ovn-nbctl lrp-add lr$i lrp-lr$i-ts1 aa:aa:aa:aa:aa:0$i 169.254.100.$i/24 + ovn-nbctl lsp-add ts1 lsp-ts1-lr$i \ + -- lsp-set-addresses lsp-ts1-lr$i router \ + -- lsp-set-type lsp-ts1-lr$i router \ + -- lsp-set-options lsp-ts1-lr$i router-port=lrp-lr$i-ts1 + + # Create static routes + ovn-nbctl lr-route-add lr$i 10.11.$i.0/24 169.254.0.1 + + # Create a src-ip route, which shouldn't be synced + ovn-nbctl --policy=src-ip --route-table=rtb1 lr-route-add lr$i 10.22.$i.0/24 169.254.0.2 +done + +for i in 1 2; do + OVS_WAIT_UNTIL([ovn_as az$i ovn-nbctl lr-route-list lr$i | grep learned]) +done + +AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl +IPv4 Routes +Route Table global: + 10.11.1.0/24 169.254.0.1 dst-ip + 10.11.2.0/24 169.254.100.2 dst-ip (learned) + +Route Table rtb1: + 10.22.1.0/24 169.254.0.2 src-ip +]) + +# move routes from global route table to rtb1 +for i in 1 2; do + ovn_as az$i ovn-nbctl lr-route-del lr$i 10.11.$i.0/24 169.254.0.1 + ovn_as az$i ovn-nbctl --route-table=rtb1 lr-route-add lr$i 10.11.$i.0/24 169.254.0.1 +done + +for i in 1 2; do + OVS_WAIT_WHILE([ovn_as az$i ovn-nbctl lr-route-list lr$i | grep learned]) +done + +# ensure route from rtb1 is not learned to any route table as route table is +# not set to TS port +AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl +IPv4 Routes +Route Table rtb1: + 10.11.1.0/24 169.254.0.1 dst-ip + 10.22.1.0/24 169.254.0.2 src-ip +]) + +# assign route table rtb1 to TS port on AZ2 and check routes are advertised to IC SB DB +check ovn_as az2 ovn-nbctl lrp-set-options lrp-lr2-ts1 route_table=rtb1 +OVS_WAIT_UNTIL([ovn-ic-sbctl find route route_table=rtb1 | grep 10.11.2.0/24]) + +# ensure route was not learned as on AZ1 TS port's LRP was not set to route table rtb1 +AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl +IPv4 Routes +Route Table rtb1: + 10.11.1.0/24 169.254.0.1 dst-ip + 10.22.1.0/24 169.254.0.2 src-ip +]) + +# set TS port's LRP to route table rtb1 to learn routes from AZ2 from rtb1 +check ovn_as az1 ovn-nbctl lrp-set-options lrp-lr1-ts1 route_table=rtb1 + +OVS_WAIT_UNTIL([ovn_as az1 ovn-nbctl --route-table=rtb1 lr-route-list lr1 | grep learned]) +AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl +IPv4 Routes +Route Table rtb1: + 10.11.1.0/24 169.254.0.1 dst-ip + 10.11.2.0/24 169.254.100.2 dst-ip (learned) + 10.22.1.0/24 169.254.0.2 src-ip +]) + +# Delete route in AZ1, AZ2's learned route should be deleted. +ovn_as az1 ovn-nbctl --route-table=rtb1 lr-route-del lr1 10.11.1.0/24 +OVS_WAIT_WHILE([ovn_as az2 ovn-nbctl --route-table=rtb1 lr-route-list lr2 | grep learned]) + +# Add the route back +ovn_as az1 ovn-nbctl --route-table=rtb1 lr-route-add lr1 10.11.1.0/24 169.254.0.1 +OVS_WAIT_UNTIL([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned]) + +# Disable route-learning for AZ1 +ovn_as az1 ovn-nbctl set nb_global . options:ic-route-learn=false +OVS_WAIT_WHILE([ovn_as az1 ovn-nbctl lr-route-list lr1 | grep learned]) +AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr1], [0], [dnl +IPv4 Routes +Route Table rtb1: + 10.11.1.0/24 169.254.0.1 dst-ip + 10.22.1.0/24 169.254.0.2 src-ip +]) + +# AZ1 should still advertise and AZ2 should still learn the route +AT_CHECK([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned], [0], [ignore]) +AT_CHECK([ovn_as az2 ovn-nbctl lr-route-list lr2], [0], [dnl +IPv4 Routes +Route Table rtb1: + 10.11.1.0/24 169.254.100.1 dst-ip (learned) + 10.11.2.0/24 169.254.0.1 dst-ip + 10.22.2.0/24 169.254.0.2 src-ip +]) + +# Disable route-advertising for AZ1 +ovn_as az1 ovn-nbctl set nb_global . options:ic-route-adv=false + +# AZ2 shouldn't have the route learned, because AZ1 have stopped advertising. +OVS_WAIT_WHILE([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned]) + +# Add default route in AZ1 +ovn_as az1 ovn-nbctl --route-table=rtb1 lr-route-add lr1 0.0.0.0/0 169.254.0.3 + +# Re-enable router-advertising & learn for AZ1 +ovn_as az1 ovn-nbctl set nb_global . options:ic-route-adv=true +ovn_as az1 ovn-nbctl set nb_global . options:ic-route-learn=true + +for i in 1 2; do + OVS_WAIT_UNTIL([ovn_as az$i ovn-nbctl lr-route-list lr$i | grep learned]) +done + +# Default route should NOT get advertised or learned, by default. +AT_CHECK([ovn-ic-sbctl find route ip_prefix="0.0.0.0/0"], [0], []) + +# Enable default route advertising in AZ1, ensure it advertised, but not learned +ovn_as az1 ovn-nbctl set nb_global . options:ic-route-adv-default=true +OVS_WAIT_UNTIL([ovn-ic-sbctl find route ip_prefix="0.0.0.0/0" route_table=rtb1 | grep 0.0.0.0]) +OVS_WAIT_WHILE([ovn_as az2 ovn-nbctl --route-table=rtb1 lr-route-list lr2 | grep learned | grep 0.0.0.0]) + +# Enable default route learning in AZ2 +ovn_as az2 ovn-nbctl set nb_global . options:ic-route-learn-default=true +OVS_WAIT_UNTIL([ovn_as az2 ovn-nbctl --route-table=rtb1 lr-route-list lr2 | grep learned | grep 0.0.0.0]) + +# Test directly connected subnet route advertising. Route should go to global route table. +ovn_as az1 ovn-nbctl lrp-add lr1 lrp-lr1-ls1 aa:aa:aa:aa:bb:01 "192.168.0.1/24" +OVS_WAIT_UNTIL([ovn-ic-sbctl find route ip_prefix="192.168.0.1/24" route_table="\"\"" | grep 192.168.0.1/24]) +OVS_WAIT_UNTIL([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned | grep 192.168]) +AT_CHECK([ovn_as az2 ovn-nbctl lr-route-list lr2], [0], [dnl +IPv4 Routes +Route Table global: + 192.168.0.0/24 169.254.100.1 dst-ip (learned) + +Route Table rtb1: + 10.11.1.0/24 169.254.100.1 dst-ip (learned) + 10.11.2.0/24 169.254.0.1 dst-ip + 10.22.2.0/24 169.254.0.2 src-ip + 0.0.0.0/0 169.254.100.1 dst-ip (learned) +]) + +# Delete the directly connected subnet from AZ1, learned route should be +# removed from AZ2. +ovn_as az1 ovn-nbctl lrp-del lrp-lr1-ls1 +OVS_WAIT_WHILE([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned | grep 192.168]) + +# Test blacklist routes +# Add back the directly connected 192.168 route. +ovn_as az1 ovn-nbctl lrp-add lr1 lrp-lr1-ls1 aa:aa:aa:aa:bb:01 "192.168.0.1/24" +OVS_WAIT_UNTIL([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned | grep 192.168]) +# Now add 10.11.0.0/16 and 192.168.0.0/16 to blacklist in AZ2. +check ovn_as az2 ovn-nbctl set nb_global . options:ic-route-blacklist="10.11.0.0/16,192.168.0.0/16" +# AZ2 shouldn't learn 192.168 route any more. +OVS_WAIT_WHILE([ovn_as az2 ovn-nbctl lr-route-list lr2 | grep learned | grep 192.168]) +# AZ1 shouldn't learn 10.11 any more. +OVS_WAIT_WHILE([ovn_as az1 ovn-nbctl lr-route-list lr1 | grep learned | grep 10.11]) +AT_CHECK([ovn_as az2 ovn-nbctl lr-route-list lr2], [0], [dnl +IPv4 Routes +Route Table rtb1: + 10.11.2.0/24 169.254.0.1 dst-ip + 10.22.2.0/24 169.254.0.2 src-ip + 0.0.0.0/0 169.254.100.1 dst-ip (learned) +]) + +OVN_CLEANUP_IC([az1], [az2]) + +AT_CLEANUP +]) + + +OVN_FOR_EACH_NORTHD([ +AT_SETUP([ovn-ic -- route sync -- multiple route tables]) + +ovn_init_ic_db +ovn-ic-nbctl ts-add ts1 + +for i in 1 2; do + ovn_start az$i + ovn_as az$i + + # Enable route learning at AZ level + ovn-nbctl set nb_global . options:ic-route-learn=true + # Enable route advertising at AZ level + ovn-nbctl set nb_global . options:ic-route-adv=true +done + +# Create new transit switches and LRs. Test topology is next: +# VPC1: +# / transit switch (ts11) \ +# logical router (lr11) - transit switch (ts12) - logical router (lr12) +# \ transit switch (ts13) / +# +# VPC2: +# / transit switch (ts21) \ +# logical router (lr21) logical router (lr22) +# \ transit switch (ts22) / +# +# each LR has one connected subnet except TS port + + +# VPC1 +# create lr11, lr12, ts11, ts12, ts13 and connect them +# assign route tables rtb1, rtb2, rtb3 to ts ports +for i in 1 2; do + ovn_as az$i + + lr=lr1$i + ovn-nbctl lr-add $lr + + for j in 1 2 3; do + ts=ts1$j + ovn-ic-nbctl --may-exist ts-add $ts + + lrp=lrp-$lr-$ts + lsp=lsp-$ts-$lr + # Create LRP and connect to TS + ovn-nbctl lrp-add $lr $lrp aa:aa:aa:aa:a$j:0$i 169.254.10$j.$i/24 + ovn-nbctl lrp-set-options $lrp route_table=rtb$j + ovn-nbctl lsp-add $ts $lsp \ + -- lsp-set-addresses $lsp router \ + -- lsp-set-type $lsp router \ + -- lsp-set-options $lsp router-port=$lrp + done +done + +# VPC2 +# create lr21, lr22, ts21, ts22 and connect them +# assign route tables rtb1, rtb2, rtb3 to ts ports +for i in 1 2; do + ovn_as az$i + + lr=lr2$i + ovn-nbctl lr-add $lr + + for j in 1 2; do + ts=ts2$j + ovn-ic-nbctl --may-exist ts-add $ts + + lrp=lrp-$lr-$ts + lsp=lsp-$ts-$lr + # Create LRP and connect to TS + ovn-nbctl lrp-add $lr $lrp aa:aa:aa:aa:a$j:0$i 169.254.10$j.$i/24 + ovn-nbctl lrp-set-options $lrp route_table=rtb$j + ovn-nbctl lsp-add $ts $lsp \ + -- lsp-set-addresses $lsp router \ + -- lsp-set-type $lsp router \ + -- lsp-set-options $lsp router-port=$lrp + done +done + +# Create directly-connected and static routes in VPC1 +ovn_as az2 ovn-nbctl lrp-add lr12 lrp-lr12 aa:aa:aa:aa:bb:01 "192.168.0.1/24" +ovn_as az2 ovn-nbctl --route-table=rtb1 lr-route-add lr12 10.10.10.0/24 192.168.0.10 +ovn_as az2 ovn-nbctl --route-table=rtb2 lr-route-add lr12 10.10.10.0/24 192.168.0.11 +ovn_as az2 ovn-nbctl --route-table=rtb3 lr-route-add lr12 10.10.10.0/24 192.168.0.12 + +# Create directly-connected route in VPC2 +ovn_as az2 ovn-nbctl lrp-add lr22 lrp-lr22 aa:aa:aa:aa:bb:01 "192.168.0.1/24" + +# Test direct routes from lr12 were learned to lr11 +AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr11 | grep 192.168 | + grep learned | awk '{print $1, $2, $5}' | sort ], [0], [dnl +192.168.0.0/24 169.254.101.2 ecmp +192.168.0.0/24 169.254.102.2 ecmp +192.168.0.0/24 169.254.103.2 ecmp +]) + +# Test static routes from lr12 rtbs rtb1,rtb2,rtb3 were learned to lr11 +AT_CHECK([ovn_as az1 ovn-nbctl --route-table=rtb1 lr-route-list lr11], [0], [dnl +IPv4 Routes +Route Table rtb1: + 10.10.10.0/24 169.254.101.2 dst-ip (learned) +]) +AT_CHECK([ovn_as az1 ovn-nbctl --route-table=rtb2 lr-route-list lr11], [0], [dnl +IPv4 Routes +Route Table rtb2: + 10.10.10.0/24 169.254.102.2 dst-ip (learned) +]) +AT_CHECK([ovn_as az1 ovn-nbctl --route-table=rtb3 lr-route-list lr11], [0], [dnl +IPv4 Routes +Route Table rtb3: + 10.10.10.0/24 169.254.103.2 dst-ip (learned) +]) + +# Test routes from lr12 didn't leak as learned to lr21 +AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr21 | grep 192.168 | sort], [0], [dnl + 192.168.0.0/24 169.254.101.2 dst-ip (learned) ecmp + 192.168.0.0/24 169.254.102.2 dst-ip (learned) ecmp +]) + +OVN_CLEANUP_IC([az1], [az2]) + +AT_CLEANUP +]) + + +OVN_FOR_EACH_NORTHD([ +AT_SETUP([ovn-ic -- route sync -- multiple route tables IPv6]) + +ovn_init_ic_db +ovn-ic-nbctl ts-add ts1 + +for i in 1 2; do + ovn_start az$i + ovn_as az$i + + # Enable route learning at AZ level + ovn-nbctl set nb_global . options:ic-route-learn=true + # Enable route advertising at AZ level + ovn-nbctl set nb_global . options:ic-route-adv=true +done + +# Create new transit switches and LRs. Test topology is next: +# VPC1: +# / transit switch (ts11) \ +# logical router (lr11) - transit switch (ts12) - logical router (lr12) +# \ transit switch (ts13) / +# +# VPC2: +# / transit switch (ts21) \ +# logical router (lr21) logical router (lr22) +# \ transit switch (ts22) / +# +# each LR has one connected subnet except TS port + + +# VPC1 +# create lr11, lr12, ts11, ts12, ts13 and connect them +# assign route tables rtb1, rtb2, rtb3 to ts ports +for i in 1 2; do + ovn_as az$i + + lr=lr1$i + ovn-nbctl lr-add $lr + + for j in 1 2 3; do + ts=ts1$j + ovn-ic-nbctl --may-exist ts-add $ts + + lrp=lrp-$lr-$ts + lsp=lsp-$ts-$lr + # Create LRP and connect to TS + ovn-nbctl lrp-add $lr $lrp aa:aa:aa:aa:a$j:0$i 2001:db8:$j::$i/64 + ovn-nbctl lrp-set-options $lrp route_table=rtb$j + ovn-nbctl lsp-add $ts $lsp \ + -- lsp-set-addresses $lsp router \ + -- lsp-set-type $lsp router \ + -- lsp-set-options $lsp router-port=$lrp + done +done + +# VPC2 +# create lr21, lr22, ts21, ts22 and connect them +# assign route tables rtb1, rtb2, rtb3 to ts ports +for i in 1 2; do + ovn_as az$i + + lr=lr2$i + ovn-nbctl lr-add $lr + + for j in 1 2; do + ts=ts2$j + ovn-ic-nbctl --may-exist ts-add $ts + + lrp=lrp-$lr-$ts + lsp=lsp-$ts-$lr + # Create LRP and connect to TS + ovn-nbctl lrp-add $lr $lrp aa:aa:aa:aa:a$j:0$i 2001:db8:$j::$i/64 + ovn-nbctl lrp-set-options $lrp route_table=rtb$j + ovn-nbctl lsp-add $ts $lsp \ + -- lsp-set-addresses $lsp router \ + -- lsp-set-type $lsp router \ + -- lsp-set-options $lsp router-port=$lrp + done +done + +# Create directly-connected and static routes in VPC1 +ovn_as az2 ovn-nbctl lrp-add lr12 lrp-lr12 aa:aa:aa:aa:bb:01 "2001:db8:200::1/64" +ovn_as az2 ovn-nbctl --route-table=rtb1 lr-route-add lr12 2001:db8:aaaa::/64 2001:db8:200::10 +ovn_as az2 ovn-nbctl --route-table=rtb2 lr-route-add lr12 2001:db8:aaaa::/64 2001:db8:200::11 +ovn_as az2 ovn-nbctl --route-table=rtb3 lr-route-add lr12 2001:db8:aaaa::/64 2001:db8:200::12 + +# Create directly-connected route in VPC2 +ovn_as az2 ovn-nbctl lrp-add lr22 lrp-lr22 aa:aa:aa:aa:bb:01 "2001:db8:200::1/64" + +# Test direct routes from lr12 were learned to lr11 +AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr11 | grep 2001:db8:200 | + grep learned | awk '{print $1, $2, $5}' | sort], [0], [dnl +2001:db8:200::/64 2001:db8:1::2 ecmp +2001:db8:200::/64 2001:db8:2::2 ecmp +2001:db8:200::/64 2001:db8:3::2 ecmp +]) + +# Test static routes from lr12 rtbs rtb1,rtb2,rtb3 were learned to lr11 +AT_CHECK([ovn_as az1 ovn-nbctl --route-table=rtb1 lr-route-list lr11], [0], [dnl +IPv6 Routes +Route Table rtb1: + 2001:db8:aaaa::/64 2001:db8:1::2 dst-ip (learned) +]) +AT_CHECK([ovn_as az1 ovn-nbctl --route-table=rtb2 lr-route-list lr11], [0], [dnl +IPv6 Routes +Route Table rtb2: + 2001:db8:aaaa::/64 2001:db8:2::2 dst-ip (learned) +]) +AT_CHECK([ovn_as az1 ovn-nbctl --route-table=rtb3 lr-route-list lr11], [0], [dnl +IPv6 Routes +Route Table rtb3: + 2001:db8:aaaa::/64 2001:db8:3::2 dst-ip (learned) +]) + +# Test routes from lr12 didn't leak as learned to lr21 +AT_CHECK([ovn_as az1 ovn-nbctl lr-route-list lr21 | grep 2001 | sort], [0], [dnl + 2001:db8:200::/64 2001:db8:1::2 dst-ip (learned) ecmp + 2001:db8:200::/64 2001:db8:2::2 dst-ip (learned) ecmp +]) + +OVN_CLEANUP_IC([az1], [az2]) + +AT_CLEANUP +])