From patchwork Fri Jun 25 23:30:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Numan Siddique X-Patchwork-Id: 1497460 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4GBYD74BKXz9sWX for ; Sat, 26 Jun 2021 09:31:17 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 45C694063E; Fri, 25 Jun 2021 23:31:14 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3fxMEfVC6w9C; Fri, 25 Jun 2021 23:31:13 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 5145C405E4; Fri, 25 Jun 2021 23:31:12 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0D2FBC001A; Fri, 25 Jun 2021 23:31:12 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id D40A2C000E for ; Fri, 25 Jun 2021 23:31:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id BB92F83BB8 for ; Fri, 25 Jun 2021 23:31:10 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pedR1QJT3tmA for ; Fri, 25 Jun 2021 23:31:09 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by smtp1.osuosl.org (Postfix) with ESMTPS id 35F0E83BB1 for ; Fri, 25 Jun 2021 23:31:08 +0000 (UTC) Received: (Authenticated sender: numans@ovn.org) by relay9-d.mail.gandi.net (Postfix) with ESMTPSA id 7586AFF802; Fri, 25 Jun 2021 23:31:05 +0000 (UTC) From: numans@ovn.org To: dev@openvswitch.org Date: Fri, 25 Jun 2021 19:30:43 -0400 Message-Id: <20210625233043.3347365-1-numans@ovn.org> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 Subject: [ovs-dev] [PATCH RFC ovn 0/1] RFC: Logical flow generation in ovn-controller X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Numan Siddique This is a an RFC patch to move the logical flow generation from ovn-northd to ovn-controller. The same patch can be found here : https://github.com/numansiddique/ovn/tree/lflows_in_controller_internal_rfc_v1 This patch doesn't move all the generation to ovn-controller. ovn-northd still does the flow generation for - ACLs/Port groups. - DHCP options - Multicast groups - And few others. Other than ACLs and Port groups it would be possible to move flow generation for the above mentioned things. But before doing all that, it is worth to evaluate if the proposed RFC makes sense. The main motivation for this RFC effort is to - Address scale issues seen. For large scale deployments ovn-northd takes lot of CPU for the computation (ovn-northd-ddlog should help here) and memory and so does Southbound ovsdb-servers. - Having a very huge southbound DB and logical flows affects the raft consenses and it requires increasing the raft election timers. - Logical flows contributes majorly to the overall south bound DB. This RFC demonstrates that it is possible for each ovn-controller to generate logical flows. These are some of the findings with my general and scale testing. Below are the test findings with a huge pre-existing Northbound database with datapath groups enabled with a size of 13 M - Southbound DB size is: * with ovn-northd-master - 35 M * with ovn-northd-proposed-rfc - 12M - Number of logical flows: * with ovn-northd-master - 78581 * with ovn-northd-proposed-rfc - 7933 - RSS Memory consumption of * ovn-northd-master - 441368 KiB * ovn-northd-proposed-rfc - 115540 KiB * ovn-controller-master - 1267716 KiB * ovn-controller-proposed-rfc - 915876 KiB * SB ovsdb-server-with-ovn-master - 612296 KiB * SB ovsdb-server-with-proposed-rfc-ovn - 134680 KiB With the scale testing of 500 fake multinode nodes and each node creating having a few port bindings claimed, the end result is almost similar. No signifcant improvements seen with the proposed RFC patch. The results are identical. I think more scale testing needs to be done to determine if the CPU usage and memory usage reduction in the ovn-northd and ovsdb-servers will have a major impact or not. Testing with a real Kubernetes/Openstack deployments would help. Few Observations - It is possible to move the flow generation to each ovn-controller. - Each ovn-controller only generates the logical flows if required i.e if the datapath is in the 'local_datapaths'. - This RFC patch do complicate ovn-controller code which has already many complicated bits. - I was expecting the scale test results to improve and the end-to-end time of a pod/VM creation would be quicker. But that is not the case, which is a disappointment. Submitting this RFC patch to get a feed back and have conversation if it is worth the effort. Numan Siddique (1): RFC: Logical flow generation in ovn-controller controller/automake.mk | 4 +- controller/binding.c | 379 +--- controller/binding.h | 13 - controller/lflow-generate.c | 179 ++ controller/lflow-generate.h | 49 + controller/lflow.c | 421 +++-- controller/lflow.h | 8 + controller/lport.c | 16 - controller/lport.h | 4 - controller/ovn-controller.c | 674 ++++++- controller/ovn-controller.h | 34 +- controller/patch.c | 1 + controller/physical.c | 58 +- controller/pinctrl.c | 18 +- lib/automake.mk | 6 +- lib/lb.c | 27 + lib/lb.h | 2 + lib/ldata.c | 895 +++++++++ lib/ldata.h | 251 +++ lib/lflow.c | 3514 +++++++++++++++++++++++++++++++++++ lib/lflow.h | 333 ++++ lib/ovn-util.c | 83 + lib/ovn-util.h | 32 + northd/ovn-northd.c | 3359 +++------------------------------ ovn-sb.ovsschema | 16 +- ovn-sb.xml | 16 + utilities/ovn-dbctl.c | 7 +- utilities/ovn-dbctl.h | 3 +- utilities/ovn-sbctl.c | 256 +++ 29 files changed, 6980 insertions(+), 3678 deletions(-) create mode 100644 controller/lflow-generate.c create mode 100644 controller/lflow-generate.h create mode 100644 lib/ldata.c create mode 100644 lib/ldata.h create mode 100644 lib/lflow.c create mode 100644 lib/lflow.h