From patchwork Wed Sep 15 16:21:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anton Ivanov X-Patchwork-Id: 1528441 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4H8lpx3Ybhz9sR4 for ; Thu, 16 Sep 2021 02:22:01 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 490994059F; Wed, 15 Sep 2021 16:21:59 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XqEt6G2iGTwu; Wed, 15 Sep 2021 16:21:58 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id 2D94E4019A; Wed, 15 Sep 2021 16:21:57 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id E354EC0011; Wed, 15 Sep 2021 16:21:56 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id C23CCC000D for ; Wed, 15 Sep 2021 16:21:55 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id B07E34019A for ; Wed, 15 Sep 2021 16:21:55 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BVL66nEZCLIa for ; Wed, 15 Sep 2021 16:21:54 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.8.0 Received: from www.kot-begemot.co.uk (ivanoab7.miniserver.com [37.128.132.42]) by smtp2.osuosl.org (Postfix) with ESMTPS id 87E1E4026A for ; Wed, 15 Sep 2021 16:21:54 +0000 (UTC) Received: from tun252.jain.kot-begemot.co.uk ([192.168.18.6] helo=jain.kot-begemot.co.uk) by www.kot-begemot.co.uk with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mQXfT-0001IV-JZ; Wed, 15 Sep 2021 16:21:51 +0000 Received: from jain.kot-begemot.co.uk ([192.168.3.3]) by jain.kot-begemot.co.uk with esmtp (Exim 4.92) (envelope-from ) id 1mQXfP-0007UV-NZ; Wed, 15 Sep 2021 17:21:49 +0100 From: anton.ivanov@cambridgegreys.com To: ovs-dev@openvswitch.org Date: Wed, 15 Sep 2021 17:21:44 +0100 Message-Id: <20210915162144.28369-1-anton.ivanov@cambridgegreys.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 X-Clacks-Overhead: GNU Terry Pratchett Cc: i.maximets@ovn.org, Anton Ivanov Subject: [ovs-dev] [OVN Patch] northd: Optimize dp/lflow postprocessing X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Anton Ivanov 1. Compute dp group hash only if there will be dp group processing. 2. Remove hmapx interim storage and related hmapx computation for single dp flows and replace it with a pre-sized hmap. Signed-off-by: Anton Ivanov --- northd/ovn-northd.c | 55 ++++++++++++++++++++++++++++----------------- ovs | 2 +- 2 files changed, 36 insertions(+), 21 deletions(-) diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c index baaddb73e..9edd1e0e4 100644 --- a/northd/ovn-northd.c +++ b/northd/ovn-northd.c @@ -13178,6 +13178,11 @@ build_lflows(struct northd_context *ctx, struct hmap *datapaths, igmp_groups, meter_groups, lbs, bfd_connections); + /* Parallel build may result in a suboptimal hash. Resize the + * hash to a correct size before doing lookups */ + + hmap_expand(&lflows); + if (hmap_count(&lflows) > max_seen_lflow_size) { max_seen_lflow_size = hmap_count(&lflows); } @@ -13185,10 +13190,22 @@ build_lflows(struct northd_context *ctx, struct hmap *datapaths, stopwatch_start(LFLOWS_DP_GROUPS_STOPWATCH_NAME, time_msec()); /* Collecting all unique datapath groups. */ struct hmap dp_groups = HMAP_INITIALIZER(&dp_groups); - struct hmapx single_dp_lflows = HMAPX_INITIALIZER(&single_dp_lflows); - struct ovn_lflow *lflow; - HMAP_FOR_EACH (lflow, hmap_node, &lflows) { - uint32_t hash = hash_int(hmapx_count(&lflow->od_group), 0); + struct hmap single_dp_lflows; + + /* Single dp_flows will never grow bigger than lflows, + * thus the two hmaps will remain the same size regardless + * of how many elements we remove from lflows and add to + * single_dp_lflows. + * Note - lflows is always sized for max_seen_lflow_size. + * If this iteration has resulted in a smaller lflow count, + * the correct sizing is from the previous ones. + */ + fast_hmap_size_for(&single_dp_lflows, max_seen_lflow_size); + + struct ovn_lflow *lflow, *next_lflow; + struct hmapx_node *node; + HMAP_FOR_EACH_SAFE (lflow, next_lflow, hmap_node, &lflows) { + uint32_t hash; struct ovn_dp_group *dpg; ovs_assert(hmapx_count(&lflow->od_group)); @@ -13196,17 +13213,24 @@ build_lflows(struct northd_context *ctx, struct hmap *datapaths, if (hmapx_count(&lflow->od_group) == 1) { /* There is only one datapath, so it should be moved out of the * group to a single 'od'. */ - const struct hmapx_node *node; HMAPX_FOR_EACH (node, &lflow->od_group) { lflow->od = node->data; break; } hmapx_clear(&lflow->od_group); + /* Logical flow should be re-hashed later to allow lookups. */ - hmapx_add(&single_dp_lflows, lflow); + hash = hmap_node_hash(&lflow->hmap_node); + /* Remove from lflows. */ + hmap_remove(&lflows, &lflow->hmap_node); + hash = ovn_logical_flow_hash_datapath(&lflow->od->sb->header_.uuid, + hash); + /* Add to single_dp_lflows. */ + hmap_insert_fast(&single_dp_lflows, &lflow->hmap_node, hash); continue; } + hash = hash_int(hmapx_count(&lflow->od_group), 0); dpg = ovn_dp_group_find(&dp_groups, &lflow->od_group, hash); if (!dpg) { dpg = xzalloc(sizeof *dpg); @@ -13216,19 +13240,11 @@ build_lflows(struct northd_context *ctx, struct hmap *datapaths, lflow->dpg = dpg; } - /* Adding datapath to the flow hash for logical flows that have only one, - * so they could be found by the southbound db record. */ - const struct hmapx_node *node; - uint32_t hash; - HMAPX_FOR_EACH (node, &single_dp_lflows) { - lflow = node->data; - hash = hmap_node_hash(&lflow->hmap_node); - hmap_remove(&lflows, &lflow->hmap_node); - hash = ovn_logical_flow_hash_datapath(&lflow->od->sb->header_.uuid, - hash); - hmap_insert(&lflows, &lflow->hmap_node, hash); - } - hmapx_destroy(&single_dp_lflows); + /* Merge multiple and single dp hashes. */ + + fast_hmap_merge(&lflows, &single_dp_lflows); + + hmap_destroy(&single_dp_lflows); /* Push changes to the Logical_Flow table to database. */ const struct sbrec_logical_flow *sbflow, *next_sbflow; @@ -13361,7 +13377,6 @@ build_lflows(struct northd_context *ctx, struct hmap *datapaths, } stopwatch_stop(LFLOWS_DP_GROUPS_STOPWATCH_NAME, time_msec()); - struct ovn_lflow *next_lflow; HMAP_FOR_EACH_SAFE (lflow, next_lflow, hmap_node, &lflows) { const char *pipeline = ovn_stage_get_pipeline_name(lflow->stage); uint8_t table = ovn_stage_get_table(lflow->stage);