From patchwork Tue Oct 1 15:17:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xavier Simonart X-Patchwork-Id: 1991570 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=RZL8y8bN; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XJ1lM2tkBz1xtg for ; Wed, 2 Oct 2024 01:17:35 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id A4092812FB; Tue, 1 Oct 2024 15:17:33 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 19QCrRFCZLhC; Tue, 1 Oct 2024 15:17:32 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=2605:bc80:3010:104::8cd3:938; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org DC94F8124F Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=RZL8y8bN Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id DC94F8124F; Tue, 1 Oct 2024 15:17:31 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 958EFC002B; Tue, 1 Oct 2024 15:17:31 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 43FC7C0894 for ; Tue, 1 Oct 2024 15:17:30 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id D48D181286 for ; Tue, 1 Oct 2024 15:17:29 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id X24JTuNc7BFu for ; Tue, 1 Oct 2024 15:17:28 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.133.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=xsimonar@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp1.osuosl.org 9D3EE811EB Authentication-Results: smtp1.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 9D3EE811EB Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id 9D3EE811EB for ; Tue, 1 Oct 2024 15:17:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1727795845; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tTL6FP/t3wQ0M491OoatBl0zmLd6CYlpWK/iG0wvzBA=; b=RZL8y8bNXZEpQWIomOpdZUSPz46uYkcrVs8QiWpJfk1l194eHmKx2yb11Re0JV0mgeEykq ej8PExnGQDH+FNoKPVydnCNCi45ydhqjx7BzrHnFvEOfrtz2wWitj5nsqLs5bDF0rzKwmU wTk8sbH3jHnJUdIB5RS58sGxrQEJNVw= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-310-8JVbWpMdPjOqOPUHkDJMnQ-1; Tue, 01 Oct 2024 11:17:23 -0400 X-MC-Unique: 8JVbWpMdPjOqOPUHkDJMnQ-1 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (unknown [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 46F2E1944DF4 for ; Tue, 1 Oct 2024 15:17:22 +0000 (UTC) Received: from wsfd-netdev90.ntdv.lab.eng.bos.redhat.com (unknown [10.6.38.135]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D05743003FD9; Tue, 1 Oct 2024 15:17:21 +0000 (UTC) From: Xavier Simonart To: xsimonar@redhat.com, dev@openvswitch.org Date: Tue, 1 Oct 2024 17:17:19 +0200 Message-Id: <20241001151719.1627801-3-xsimonar@redhat.com> In-Reply-To: <20241001151719.1627801-1-xsimonar@redhat.com> References: <20241001134023.136629-2-xsimonar@redhat.com> <20241001151719.1627801-1-xsimonar@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH ovn v2 2/2] controller: Avoid potential 100% cpu when ports are postponed. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" When two chassis claim the same port, if a port is in the MARK_UP state when the other chassis claims it, then ovn-controller was sometimes using 100% cpu, as if-status was fighting to add ovn-install (as in the MARK_UP state) and binding was fighting to remove it (as the port was bound to the other chassis). Signed-off-by: Xavier Simonart Reviewed-by: Ales Musil --- v2: rebase on origin/main for scapy fix. --- controller/binding.c | 15 +++++---------- controller/if-status.c | 27 +++++++++++++++++++++------ controller/if-status.h | 3 +-- tests/ovn.at | 3 +++ 4 files changed, 30 insertions(+), 18 deletions(-) diff --git a/controller/binding.c b/controller/binding.c index 3b9bc8620..8c6f5aabe 100644 --- a/controller/binding.c +++ b/controller/binding.c @@ -1608,8 +1608,7 @@ consider_vif_lport_(const struct sbrec_port_binding *pb, /* Release the lport if there is no lbinding. */ if (lbinding_set && !can_bind) { if_status_mgr_remove_ovn_installed(b_ctx_out->if_mgr, - b_lport->lbinding->iface->name, - &b_lport->lbinding->iface->header_.uuid); + b_lport->lbinding->iface); } if (!lbinding_set || !can_bind) { @@ -1626,8 +1625,7 @@ consider_vif_lport_(const struct sbrec_port_binding *pb, pb->logical_port)) { update_lport_tracking(pb, b_ctx_out->tracked_dp_bindings, false); if_status_mgr_remove_ovn_installed(b_ctx_out->if_mgr, - b_lport->lbinding->iface->name, - &b_lport->lbinding->iface->header_.uuid); + b_lport->lbinding->iface); } return true; @@ -2062,8 +2060,7 @@ build_local_bindings(struct binding_ctx_in *b_ctx_in, * This can happen if iface-id was removed as we recompute. */ if_status_mgr_remove_ovn_installed(b_ctx_out->if_mgr, - iface_rec->name, - &iface_rec->header_.uuid); + iface_rec); } } } @@ -2491,8 +2488,7 @@ consider_iface_release(const struct ovsrec_interface *iface_rec, } if (lbinding->iface && lbinding->iface->name) { if_status_mgr_remove_ovn_installed(b_ctx_out->if_mgr, - lbinding->iface->name, - &lbinding->iface->header_.uuid); + lbinding->iface); } } else if (b_lport && b_lport->type == LP_LOCALPORT) { @@ -2824,8 +2820,7 @@ handle_deleted_vif_lport(const struct sbrec_port_binding *pb, handle_deleted_lport(pb, b_ctx_in, b_ctx_out); if (lbinding && lbinding->iface && lbinding->iface->name) { if_status_mgr_remove_ovn_installed(b_ctx_out->if_mgr, - lbinding->iface->name, - &lbinding->iface->header_.uuid); + lbinding->iface); } return true; } diff --git a/controller/if-status.c b/controller/if-status.c index ada78a18b..dc7b66ee2 100644 --- a/controller/if-status.c +++ b/controller/if-status.c @@ -71,6 +71,8 @@ enum if_state { * but not yet marked "up" in the binding module (in * SB and OVS databases). */ + OIF_UNINSTALLED, /* Interface from whom ovn-install must be removed + */ OIF_INSTALLED, /* Interface flows programmed in OVS and binding * marked "up" in the binding module. */ @@ -89,6 +91,7 @@ static const char *if_state_names[] = { [OIF_REM_OLD_OVN_INST] = "REM_OLD_OVN_INST", [OIF_MARK_UP] = "MARK_UP", [OIF_MARK_DOWN] = "MARK_DOWN", + [OIF_UNINSTALLED] = "UNINSTALLED", [OIF_INSTALLED] = "INSTALLED", [OIF_UPDATE_PORT] = "UPDATE_PORT", }; @@ -326,6 +329,7 @@ if_status_mgr_claim_iface(struct if_status_mgr *mgr, /* Nothing to do here. */ break; case OIF_INSTALLED: + case OIF_UNINSTALLED: case OIF_MARK_DOWN: case OIF_UPDATE_PORT: ovs_iface_set_state(mgr, iface, OIF_CLAIMED); @@ -362,6 +366,7 @@ if_status_mgr_release_iface(struct if_status_mgr *mgr, const char *iface_id) case OIF_REM_OLD_OVN_INST: case OIF_MARK_UP: case OIF_INSTALLED: + case OIF_UNINSTALLED: /* Properly mark interfaces "down" if their flows were already * programmed in OVS. */ @@ -402,6 +407,7 @@ if_status_mgr_delete_iface(struct if_status_mgr *mgr, const char *iface_id, case OIF_REM_OLD_OVN_INST: case OIF_MARK_UP: case OIF_INSTALLED: + case OIF_UNINSTALLED: /* Properly mark interfaces "down" if their flows were already * programmed in OVS. */ @@ -634,13 +640,22 @@ if_status_mgr_update(struct if_status_mgr *mgr, void if_status_mgr_remove_ovn_installed(struct if_status_mgr *mgr, - const char *name, - const struct uuid *uuid) + const struct ovsrec_interface *iface_rec) { - VLOG_DBG("Adding %s to list of interfaces for which to remove " - "ovn-installed", name); - if (!shash_find_data(&mgr->ovn_uninstall_hash, name)) { - add_to_ovn_uninstall_hash(mgr, name, uuid); + if (!shash_find_data(&mgr->ovn_uninstall_hash, iface_rec->name)) { + VLOG_DBG("Adding %s to list of interfaces for which to remove " + "ovn-installed", iface_rec->name); + add_to_ovn_uninstall_hash(mgr, iface_rec->name, + &iface_rec->header_.uuid); + } + + /* Move out of MARK_UP state which would add ovn-install back. */ + const char *iface_id = smap_get(&iface_rec->external_ids, "iface-id"); + if (iface_id) { + struct ovs_iface *iface = shash_find_data(&mgr->ifaces, iface_id); + if (iface && iface->state == OIF_MARK_UP) { + ovs_iface_set_state(mgr, iface, OIF_UNINSTALLED); + } } } diff --git a/controller/if-status.h b/controller/if-status.h index 4ae5ad481..eb91b62fd 100644 --- a/controller/if-status.h +++ b/controller/if-status.h @@ -61,8 +61,7 @@ bool if_status_handle_claims(struct if_status_mgr *mgr, const struct sbrec_port_binding_table *pb_table, bool sb_readonly); void if_status_mgr_remove_ovn_installed(struct if_status_mgr *mgr, - const char *name, - const struct uuid *uuid); + const struct ovsrec_interface *iface_rec); uint16_t if_status_mgr_iface_get_mtu(const struct if_status_mgr *mgr, const char *iface_id); bool if_status_mgr_iface_update(const struct if_status_mgr *mgr, diff --git a/tests/ovn.at b/tests/ovn.at index 826b52051..f52d7f3e2 100644 --- a/tests/ovn.at +++ b/tests/ovn.at @@ -16552,6 +16552,9 @@ max_claims=20 AT_CHECK([test "${hv1_claims}" -le "${max_claims}"], [0], []) AT_CHECK([test "${hv2_claims}" -le "${max_claims}"], [0], []) +n_uninstalled=$(as hv1 grep -c 'Removing iface vif ovn-installed in' hv1/ovn-controller.log) +echo "$n_uninstalled times 'Removing iface vif ovn-installed'" +AT_CHECK([test "${n_uninstalled}" -le "10"], [0], []) check ovn-nbctl --wait=hv lsp-del lsp0 CHECK_AFTER_RECOMPUTE([hv1], [hv1])