From patchwork Wed Jul 3 12:20:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ales Musil X-Patchwork-Id: 1956215 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=L0X92cKG; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4WDf4y14xsz1xqb for ; Wed, 3 Jul 2024 22:20:49 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 69BCC41AD9; Wed, 3 Jul 2024 12:20:43 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 3HdSKoNntFzm; Wed, 3 Jul 2024 12:20:40 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=2605:bc80:3010:104::8cd3:938; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 6715440227 Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=L0X92cKG Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 6715440227; Wed, 3 Jul 2024 12:20:40 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1BA9DC09E7; Wed, 3 Jul 2024 12:20:40 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0CD70C0611 for ; Wed, 3 Jul 2024 12:20:39 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id E2B8D41764 for ; Wed, 3 Jul 2024 12:20:38 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 5EbuL5_Opjen for ; Wed, 3 Jul 2024 12:20:37 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.133.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=amusil@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp4.osuosl.org 737234175F Authentication-Results: smtp4.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 737234175F Authentication-Results: smtp4.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=L0X92cKG Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id 737234175F for ; Wed, 3 Jul 2024 12:20:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1720009235; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=mPPYusy5WbuZ5X+DXMDTjK3r83otyX6wojvKjOqc+C8=; b=L0X92cKGQbPZc8F/qopRkWEkQeC+WHXEuRwhMKHmykXLUfDkiJTWOnPkDw8IrdRq8bmnwh ZGCJEs6befunGIRSX5zgjACNGdRPe/ijs+/4UFWCMF4GcyaxEBrXFbAsBBRyMrnlvgAGKX fUp8+YhiQZQCMujyTkdyaBN++DMbbWk= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-42-qkGeCA9hMUufcGeYKslczQ-1; Wed, 03 Jul 2024 08:20:32 -0400 X-MC-Unique: qkGeCA9hMUufcGeYKslczQ-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A86F619560B2 for ; Wed, 3 Jul 2024 12:20:31 +0000 (UTC) Received: from amusil.brq.redhat.com (unknown [10.43.17.32]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id B1384195607C; Wed, 3 Jul 2024 12:20:30 +0000 (UTC) From: Ales Musil To: dev@openvswitch.org Date: Wed, 3 Jul 2024 14:20:29 +0200 Message-ID: <20240703122029.1265688-1-amusil@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH ovn] northd: Detect if SB LB was removed and re-sync that row. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" When someone removes SB LB it won't be detected until the next recompute of the "sync_to_sb_lb" node. This will cause traffic disruptions in case of hairpin as the flows directly depend on the SB LB entries. Add check to trigger recompute when we detect that the row is missing in SB, but still present in NB. Reported-at: https://issues.redhat.com/browse/FDP-682 Signed-off-by: Ales Musil --- northd/en-sync-sb.c | 38 +++++++++++++++++++++++++++++++++----- tests/ovn-macros.at | 2 +- tests/ovn-northd.at | 38 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 72 insertions(+), 6 deletions(-) diff --git a/northd/en-sync-sb.c b/northd/en-sync-sb.c index 9bd8a1fc6..12dbd139e 100644 --- a/northd/en-sync-sb.c +++ b/northd/en-sync-sb.c @@ -20,6 +20,7 @@ /* OVS includes. */ #include "lib/svec.h" +#include "lib/uuidset.h" #include "openvswitch/util.h" /* OVN includes. */ @@ -232,6 +233,7 @@ struct sb_lb_table { struct hmap entries; /* Stores struct sb_lb_record. */ struct hmap ls_dp_groups; struct hmap lr_dp_groups; + struct uuidset sb_entries; }; struct ed_type_sync_to_sb_lb_data { @@ -347,16 +349,29 @@ sync_to_sb_lb_northd_handler(struct engine_node *node, void *data_) } bool -sync_to_sb_lb_sb_load_balancer(struct engine_node *node, void *data OVS_UNUSED) +sync_to_sb_lb_sb_load_balancer(struct engine_node *node, void *data_) { const struct sbrec_load_balancer_table *sb_load_balancer_table = EN_OVSDB_GET(engine_get_input("SB_load_balancer", node)); + struct ed_type_sync_to_sb_lb_data *data = data_; - /* The only reason to handle SB.Load_Balancer updates is to detect + /* The reasons to handle SB.Load_Balancer updates is to detect * spurious records being created in clustered databases due to - * lack of indexing on the SB.Load_Balancer table. All other changes - * are valid and performed by northd, the only write-client for - * this table. */ + * lack of indexing on the SB.Load_Balancer table. The other reason might + * be when someone removes the SB row while the NB row is still valid. + * All other changes are valid and performed by northd. */ + + const struct sbrec_load_balancer *sb_lb; + SBREC_LOAD_BALANCER_TABLE_FOR_EACH_TRACKED (sb_lb, + sb_load_balancer_table) { + if (sbrec_load_balancer_is_deleted(sb_lb) && + uuidset_find(&data->sb_lbs.sb_entries, &sb_lb->header_.uuid)) { + VLOG_WARN("A SB LB for \"%s\" is deleted but the NB LB entry " + "still exists.", sb_lb->name); + return false; + } + } + if (check_sb_lb_duplicates(sb_load_balancer_table)) { return false; } @@ -626,6 +641,7 @@ sb_lb_table_init(struct sb_lb_table *sb_lbs) hmap_init(&sb_lbs->entries); ovn_dp_groups_init(&sb_lbs->ls_dp_groups); ovn_dp_groups_init(&sb_lbs->lr_dp_groups); + uuidset_init(&sb_lbs->sb_entries); } static void @@ -638,6 +654,7 @@ sb_lb_table_clear(struct sb_lb_table *sb_lbs) ovn_dp_groups_clear(&sb_lbs->ls_dp_groups); ovn_dp_groups_clear(&sb_lbs->lr_dp_groups); + uuidset_clear(&sb_lbs->sb_entries); } static void @@ -647,6 +664,7 @@ sb_lb_table_destroy(struct sb_lb_table *sb_lbs) hmap_destroy(&sb_lbs->entries); ovn_dp_groups_destroy(&sb_lbs->ls_dp_groups); ovn_dp_groups_destroy(&sb_lbs->lr_dp_groups); + uuidset_destroy(&sb_lbs->sb_entries); } static struct sb_lb_record * @@ -693,6 +711,8 @@ sb_lb_table_build_and_sync( const char *nb_lb_uuid = smap_get(&sbrec_lb->external_ids, "lb_id"); struct uuid lb_uuid; if (!nb_lb_uuid || !uuid_from_string(&lb_uuid, nb_lb_uuid)) { + uuidset_find_and_delete(&sb_lbs->sb_entries, + &sbrec_lb->header_.uuid); sbrec_load_balancer_delete(sbrec_lb); continue; } @@ -711,6 +731,8 @@ sb_lb_table_build_and_sync( hmap_insert(&sb_lbs->entries, &sb_lb->key_node, uuid_hash(&sb_lb->lb_dps->lb->nlb->header_.uuid)); } else { + uuidset_find_and_delete(&sb_lbs->sb_entries, + &sbrec_lb->header_.uuid); sbrec_load_balancer_delete(sbrec_lb); } } @@ -770,6 +792,8 @@ sync_sb_lb_record(struct sb_lb_record *sb_lb, sbrec_lr_dp_group = sbrec_lb->lr_datapath_group; } + uuidset_insert(&sb_lbs->sb_entries, &sb_lb->sb_uuid); + if (lb_dps->n_nb_ls) { sb_lb->ls_dpg = ovn_dp_group_get(&sb_lbs->ls_dp_groups, lb_dps->n_nb_ls, @@ -931,6 +955,8 @@ sync_changed_lbs(struct sb_lb_table *sb_lbs, sbrec_load_balancer_table_get_for_uuid(sb_lb_table, &sb_lb->sb_uuid); if (sbrec_lb) { + uuidset_find_and_delete(&sb_lbs->sb_entries, + &sbrec_lb->header_.uuid); sbrec_load_balancer_delete(sbrec_lb); } @@ -965,6 +991,8 @@ sync_changed_lbs(struct sb_lb_table *sb_lbs, sbrec_load_balancer_table_get_for_uuid(sb_lb_table, &sb_lb->sb_uuid); if (sbrec_lb) { + uuidset_find_and_delete(&sb_lbs->sb_entries, + &sbrec_lb->header_.uuid); sbrec_load_balancer_delete(sbrec_lb); } diff --git a/tests/ovn-macros.at b/tests/ovn-macros.at index 47ada5c70..65379ea57 100644 --- a/tests/ovn-macros.at +++ b/tests/ovn-macros.at @@ -249,7 +249,7 @@ ovn_start_northd() { local name=${d_prefix}northd${suffix} echo "${prefix}starting $name" test -d "$ovs_base/$name" || mkdir "$ovs_base/$name" - as $name start_daemon ovn-northd $northd_args -vjsonrpc \ + as $name start_daemon ovn-northd $northd_args \ --ovnnb-db=$OVN_NB_DB --ovnsb-db=$OVN_SB_DB } diff --git a/tests/ovn-northd.at b/tests/ovn-northd.at index a389d1988..74726f449 100644 --- a/tests/ovn-northd.at +++ b/tests/ovn-northd.at @@ -12721,3 +12721,41 @@ AT_CHECK([ovn-sbctl dump-flows lr | grep lr_in_dnat | ovn_strip_lflows], [0], [d AT_CLEANUP ]) + +OVN_FOR_EACH_NORTHD_NO_HV([ +AT_SETUP([LB re-sync]) +ovn_start + +check ovn-sbctl chassis-add hv1 geneve 127.0.0.1 + +check ovn-nbctl lr-add lr -- \ + lrp-add lr lr-ls 02:ac:10:01:00:01 172.16.1.1/24 + +check ovn-nbctl ls-add ls -- \ + lsp-add ls ls-lr -- \ + lsp-set-type ls-lr router -- \ + lsp-set-addresses ls-lr router -- \ + lsp-set-options ls-lr router-port=lr-ls + +check ovn-nbctl lb-add lb 172.16.10.10 192.168.10.10 +check ovn-nbctl --wait=sb lr-lb-add lr lb + +check_row_count Load_Balancer 1 + +check ovn-sbctl --all destroy Load_Balancer +check ovn-nbctl --wait=sb sync +check_row_count Load_Balancer 1 + +check ovn-nbctl --wait=sb lb-del lb +wait_row_count Load_Balancer 0 + +check ovn-nbctl lb-add lb 172.16.10.10 192.168.10.10 +check ovn-nbctl --wait=sb lr-lb-add lr lb +check_row_count Load_Balancer 1 + +check ovn-sbctl --all destroy Load_Balancer +check ovn-nbctl --wait=sb sync +check_row_count Load_Balancer 1 + +AT_CLEANUP +])