From patchwork Mon Sep 11 15:59:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1832346 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RkryX0x7mz1yhZ for ; Tue, 12 Sep 2023 02:00:04 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 4BC9181E9D; Mon, 11 Sep 2023 16:00:02 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 4BC9181E9D X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gf0oxjXazbpG; Mon, 11 Sep 2023 16:00:01 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 5267181B83; Mon, 11 Sep 2023 16:00:00 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 5267181B83 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 361ACC0071; Mon, 11 Sep 2023 16:00:00 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id DBFBFC0032 for ; Mon, 11 Sep 2023 15:59:58 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id A5CFD81B14 for ; Mon, 11 Sep 2023 15:59:58 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org A5CFD81B14 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QgcGkbZIIvIq for ; Mon, 11 Sep 2023 15:59:58 +0000 (UTC) Received: from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net [217.70.183.201]) by smtp1.osuosl.org (Postfix) with ESMTPS id 8340281B83 for ; Mon, 11 Sep 2023 15:59:57 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 8340281B83 Received: by mail.gandi.net (Postfix) with ESMTPSA id 432221BF214; Mon, 11 Sep 2023 15:59:35 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Mon, 11 Sep 2023 11:59:30 -0400 Message-ID: <20230911155930.179283-1-numans@ovn.org> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 X-GND-Sasl: numans@ovn.org Subject: [ovs-dev] [PATCH ovn v7 0/4] northd: I-P for load balancer and lb groups X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique This patch series adds the support to handle load balancer and load balancer group changes incrementally in the "northd" engine node. Changes to logical switches and router's load balancer and load balancer group columns are also handled incrementally provided other columns do not change. v6 of the series also included lflow I-P handling. But v7 drops these patches as there are some concerns with the obj dep mgr usage. lflow I-P handling patches will be submitted separately. Below are the scale testing results done with these patches applied using ovn-heater. The test ran the scenario - ocp-500-density-heavy.yml [1]. With these patches applied (with load balancer I-P handling only in northd engine node) the resuts are: ------------------------------------------------------------------------------------------------------------------------------------------------------- Min (s) Median (s) 90%ile (s) 99%ile (s) Max (s) Mean (s) Total (s) Count Failed ------------------------------------------------------------------------------------------------------------------------------------------------------- Iteration Total 0.131363 1.189994 3.213526 4.308134 4.394562 1.385713 173.214153 125 0 Namespace.add_ports 0.005176 0.005611 0.006476 0.019867 0.024206 0.006058 0.757188 125 0 WorkerNode.bind_port 0.033776 0.046343 0.054414 0.061719 0.063613 0.046815 11.703773 250 0 WorkerNode.ping_port 0.005156 0.006959 2.044939 3.655328 4.241496 0.627103 156.775832 250 0 ------------------------------------------------------------------------------------------------------------------------------------------------------- The results with the present main (Result 3) are: ------------------------------------------------------------------------------------------------------------------------------------------------------- Min (s) Median (s) 90%ile (s) 99%ile (s) Max (s) Mean (s) Total (s) Count Failed ------------------------------------------------------------------------------------------------------------------------------------------------------- Iteration Total 3.233795 4.364926 5.400982 6.412803 7.409757 4.792270 599.033790 125 0 Namespace.add_ports 0.005230 0.006564 0.007379 0.019060 0.037490 0.007223 0.902930 125 0 WorkerNode.bind_port 0.033864 0.044052 0.049608 0.054849 0.056196 0.044005 11.001231 250 0 WorkerNode.ping_port 0.005334 2.060477 5.222422 6.267332 7.284001 2.323020 580.754964 250 0 ------------------------------------------------------------------------------------------------------------------------------------------------------- v6 -> v7 ------- * First 4 patches of v6 are merged in main and branch-23.09 and patches 9 to 16 are dropped. * v7 only has 4 patches now. * Addressed review comments. There is only one handler for lb_data engine input in northd engine node - northd_handle_lb_data_changes(). In v6 and earlier there were 2 handle functions - northd_handle_lb_data_changes_pre_od() and northd_handle_lb_data_changes_post_od(). v5 -> v6 ------- * Rebased. Added 2 more patches (p15 and p16) for LR NAT I-P handling. v4 -> v5 ------- * 6 new patches are added to the series which handles the LB changes in the lflow engine node. v3 -> v4 ------- * Covered more test scearios. * Found few issues and fixed them. v3 was not handling the scenario of a vip getting added or removed from a load balancer. v2 -> v3 -------- * v2 was very inefficient in handling the load balancer group changes and in associating the load balancers of the lb group to the datapaths. This was the main reason for the regression in the full recompute time taken. v3 addressed these by more efficiently handling the lb group changes incrementally. Numan Siddique (4): northd: Handle load balancer changes for a logical switch. northd: Handle load balancer group changes for a logical switch. northd: Sync SB Port bindings NAT column in a separate engine node. northd: Handle load balancer/group changes for a logical router. lib/lb.c | 51 ++- lib/lb.h | 9 + northd/en-lb-data.c | 431 ++++++++++++++++++++++++- northd/en-lb-data.h | 35 +++ northd/en-lflow.c | 6 + northd/en-northd.c | 26 +- northd/en-northd.h | 1 + northd/en-sync-sb.c | 52 +++ northd/en-sync-sb.h | 5 + northd/inc-proc-northd.c | 15 +- northd/northd.c | 660 ++++++++++++++++++++++++++++++--------- northd/northd.h | 9 +- tests/ovn-northd.at | 145 +++++---- 13 files changed, 1224 insertions(+), 221 deletions(-)