From patchwork Mon Mar 7 12:39:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valerio X-Patchwork-Id: 1602136 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=C7GWmXZb; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4KByhX74mSz9sFr for ; Mon, 7 Mar 2022 23:39:40 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id C2F9260E6A; Mon, 7 Mar 2022 12:39:38 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ufnnEcgbPa2M; Mon, 7 Mar 2022 12:39:37 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id C3F0E60E5B; Mon, 7 Mar 2022 12:39:36 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8DBF8C0012; Mon, 7 Mar 2022 12:39:36 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 19651C0012 for ; Mon, 7 Mar 2022 12:39:35 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id E568981DAB for ; Mon, 7 Mar 2022 12:39:33 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp1.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NBTNx2paOS6O for ; Mon, 7 Mar 2022 12:39:32 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id 8BD1981D9F for ; Mon, 7 Mar 2022 12:39:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1646656771; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1TodbPvq/GS/CsfCtL/3GaF9REw8GAfWRLnOviVraPA=; b=C7GWmXZbRddJexZ+efI5b/VkuO0PjQoGAQxYNo9/N4ZgsNRV6gjRm4fa7KDqlNqkDyVrW/ kd1lJ2as7ElQ0VB/vCKtx7yM/KSFwsh8hyH8JoqL9dkjS0wjFIE893MiZenxBV8nR7Dfk7 PoYFacKrV3rKldI+frLajebSWTX6qSg= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-307-87sHWvWeNIKXtJHOkmq7IA-1; Mon, 07 Mar 2022 07:39:29 -0500 X-MC-Unique: 87sHWvWeNIKXtJHOkmq7IA-1 Received: by mail-wr1-f71.google.com with SMTP id f9-20020a5d58e9000000b001f0247e5e96so4510305wrd.15 for ; Mon, 07 Mar 2022 04:39:29 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=1TodbPvq/GS/CsfCtL/3GaF9REw8GAfWRLnOviVraPA=; b=dipVj2NIGBWMtO7sxgytrKu4A+GXTnShKQi5NIwlL2akxbJb/GmDHQvP1wuv1b8Wgn HRGAEOb9bXmr4hlFp6MkQ1vw9z++NNBq2LjOnY4RU9YesYhJTewIrcYEw0boDn7s0GFD wvtXv61ris7QA1VkuL2vNfN2qs/7GPvNL5zlzNDz8mHOpK4V9YbL5MBs4D3dnqKCUz6B C42t3kYopqT4NBQhHH7KFz7Qa81ceKOQku5vteIiCydwsSJm7KNB/d2eQIzZnwIRd+x5 cMdNTLAA4uUOhRHiXJTP9CeLNBbWSxv9m5kJuFlffRVhEWZ07Cpztf1M0zQ1U6XZc+MW xEsA== X-Gm-Message-State: AOAM530prcJsbt/TaNbMgj9+ghYVkho1wHOmgSSLSRlKbR1Qhk2u1Fqe voRI2lFCv1wvPemZpOUFvx8qgNU/aRwQkSIzg0w7Y+Sy1tfGRHkIV8g3sL9ShEg0r84DqhLvXZ3 B9KvhGnTqKf63wOJMcu88ffDQ5DWGTEaPVxFU8NQw+8m2j9w7TMAU8UMTqyziGx7L X-Received: by 2002:a05:600c:5115:b0:389:bd64:85ab with SMTP id o21-20020a05600c511500b00389bd6485abmr270160wms.32.1646656768068; Mon, 07 Mar 2022 04:39:28 -0800 (PST) X-Google-Smtp-Source: ABdhPJwgFsdYeO4RjP80+TDRsjUdkfxxv3jdTdlPRf8A771okIQGigHiazoQQzpct8lDwLQypqO0QQ== X-Received: by 2002:a05:600c:5115:b0:389:bd64:85ab with SMTP id o21-20020a05600c511500b00389bd6485abmr270129wms.32.1646656767702; Mon, 07 Mar 2022 04:39:27 -0800 (PST) Received: from localhost (net-93-66-41-83.cust.vodafonedsl.it. [93.66.41.83]) by smtp.gmail.com with ESMTPSA id d8-20020a056000114800b001f045f3435asm18183499wrx.108.2022.03.07.04.39.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Mar 2022 04:39:27 -0800 (PST) From: Paolo Valerio To: dev@openvswitch.org Date: Mon, 07 Mar 2022 13:39:26 +0100 Message-ID: <164665676191.3966098.14801639154172658052.stgit@fed.void> In-Reply-To: <164665673570.3966098.2589967971949905134.stgit@fed.void> References: <164665673570.3966098.2589967971949905134.stgit@fed.void> User-Agent: StGit/1.1 MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pvalerio@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: fbl@redhat.com, i.maximets@ovn.org Subject: [ovs-dev] [PATCH v2 2/6] conntrack-tp: Use a cmap to store timeout policies X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Gaetan Rivet Multiple lookups are done to stored timeout policies, each time blocking the global 'ct_lock'. This is usually not necessary and it should be acceptable to get policy updates slightly delayed (by one RCU sync at most). Using a CMAP reduces multiple lock taking and releasing in the connection insertion path. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Acked-by: William Tu Signed-off-by: Paolo Valerio --- lib/conntrack-private.h | 2 +- lib/conntrack-tp.c | 54 ++++++++++++++++++++++++++--------------------- lib/conntrack.c | 9 +++++--- lib/conntrack.h | 2 +- 4 files changed, 38 insertions(+), 29 deletions(-) diff --git a/lib/conntrack-private.h b/lib/conntrack-private.h index d9461b811..34c688821 100644 --- a/lib/conntrack-private.h +++ b/lib/conntrack-private.h @@ -193,7 +193,7 @@ struct conntrack { struct cmap conns OVS_GUARDED; struct ovs_list exp_lists[N_CT_TM] OVS_GUARDED; struct cmap zone_limits OVS_GUARDED; - struct hmap timeout_policies OVS_GUARDED; + struct cmap timeout_policies OVS_GUARDED; uint32_t hash_basis; /* Salt for hashing a connection key. */ pthread_t clean_thread; /* Periodically cleans up connection tracker. */ struct latch clean_thread_exit; /* To destroy the 'clean_thread'. */ diff --git a/lib/conntrack-tp.c b/lib/conntrack-tp.c index a586d3a8d..c2245038b 100644 --- a/lib/conntrack-tp.c +++ b/lib/conntrack-tp.c @@ -47,14 +47,15 @@ static unsigned int ct_dpif_netdev_tp_def[] = { }; static struct timeout_policy * -timeout_policy_lookup(struct conntrack *ct, int32_t tp_id) +timeout_policy_lookup_protected(struct conntrack *ct, int32_t tp_id) OVS_REQUIRES(ct->ct_lock) { struct timeout_policy *tp; uint32_t hash; hash = hash_int(tp_id, ct->hash_basis); - HMAP_FOR_EACH_IN_BUCKET (tp, node, hash, &ct->timeout_policies) { + CMAP_FOR_EACH_WITH_HASH_PROTECTED (tp, node, hash, + &ct->timeout_policies) { if (tp->policy.id == tp_id) { return tp; } @@ -62,20 +63,25 @@ timeout_policy_lookup(struct conntrack *ct, int32_t tp_id) return NULL; } -struct timeout_policy * -timeout_policy_get(struct conntrack *ct, int32_t tp_id) +static struct timeout_policy * +timeout_policy_lookup(struct conntrack *ct, int32_t tp_id) { struct timeout_policy *tp; + uint32_t hash; - ovs_mutex_lock(&ct->ct_lock); - tp = timeout_policy_lookup(ct, tp_id); - if (!tp) { - ovs_mutex_unlock(&ct->ct_lock); - return NULL; + hash = hash_int(tp_id, ct->hash_basis); + CMAP_FOR_EACH_WITH_HASH (tp, node, hash, &ct->timeout_policies) { + if (tp->policy.id == tp_id) { + return tp; + } } + return NULL; +} - ovs_mutex_unlock(&ct->ct_lock); - return tp; +struct timeout_policy * +timeout_policy_get(struct conntrack *ct, int32_t tp_id) +{ + return timeout_policy_lookup(ct, tp_id); } static void @@ -125,27 +131,30 @@ timeout_policy_create(struct conntrack *ct, init_default_tp(tp, tp_id); update_existing_tp(tp, new_tp); hash = hash_int(tp_id, ct->hash_basis); - hmap_insert(&ct->timeout_policies, &tp->node, hash); + cmap_insert(&ct->timeout_policies, &tp->node, hash); } static void timeout_policy_clean(struct conntrack *ct, struct timeout_policy *tp) OVS_REQUIRES(ct->ct_lock) { - hmap_remove(&ct->timeout_policies, &tp->node); - free(tp); + uint32_t hash = hash_int(tp->policy.id, ct->hash_basis); + cmap_remove(&ct->timeout_policies, &tp->node, hash); + ovsrcu_postpone(free, tp); } static int -timeout_policy_delete__(struct conntrack *ct, uint32_t tp_id) +timeout_policy_delete__(struct conntrack *ct, uint32_t tp_id, + bool warn_on_error) OVS_REQUIRES(ct->ct_lock) { + struct timeout_policy *tp; int err = 0; - struct timeout_policy *tp = timeout_policy_lookup(ct, tp_id); + tp = timeout_policy_lookup_protected(ct, tp_id); if (tp) { timeout_policy_clean(ct, tp); - } else { + } else if (warn_on_error) { VLOG_WARN_RL(&rl, "Failed to delete a non-existent timeout " "policy: id=%d", tp_id); err = ENOENT; @@ -159,7 +168,7 @@ timeout_policy_delete(struct conntrack *ct, uint32_t tp_id) int err; ovs_mutex_lock(&ct->ct_lock); - err = timeout_policy_delete__(ct, tp_id); + err = timeout_policy_delete__(ct, tp_id, true); ovs_mutex_unlock(&ct->ct_lock); return err; } @@ -170,7 +179,7 @@ timeout_policy_init(struct conntrack *ct) { struct timeout_policy tp; - hmap_init(&ct->timeout_policies); + cmap_init(&ct->timeout_policies); /* Create default timeout policy. */ memset(&tp, 0, sizeof tp); @@ -182,14 +191,11 @@ int timeout_policy_update(struct conntrack *ct, struct timeout_policy *new_tp) { - int err = 0; uint32_t tp_id = new_tp->policy.id; + int err = 0; ovs_mutex_lock(&ct->ct_lock); - struct timeout_policy *tp = timeout_policy_lookup(ct, tp_id); - if (tp) { - err = timeout_policy_delete__(ct, tp_id); - } + timeout_policy_delete__(ct, tp_id, false); timeout_policy_create(ct, new_tp); ovs_mutex_unlock(&ct->ct_lock); return err; diff --git a/lib/conntrack.c b/lib/conntrack.c index bb605bac9..13a1dd519 100644 --- a/lib/conntrack.c +++ b/lib/conntrack.c @@ -542,10 +542,13 @@ conntrack_destroy(struct conntrack *ct) cmap_destroy(&ct->zone_limits); struct timeout_policy *tp; - HMAP_FOR_EACH_POP (tp, node, &ct->timeout_policies) { - free(tp); + CMAP_FOR_EACH (tp, node, &ct->timeout_policies) { + uint32_t hash = hash_int(tp->policy.id, ct->hash_basis); + + cmap_remove(&ct->timeout_policies, &tp->node, hash); + ovsrcu_postpone(free, tp); } - hmap_destroy(&ct->timeout_policies); + cmap_destroy(&ct->timeout_policies); ovs_mutex_unlock(&ct->ct_lock); ovs_mutex_destroy(&ct->ct_lock); diff --git a/lib/conntrack.h b/lib/conntrack.h index 58b181834..b064abc9f 100644 --- a/lib/conntrack.h +++ b/lib/conntrack.h @@ -113,7 +113,7 @@ struct conntrack_zone_limit { }; struct timeout_policy { - struct hmap_node node; + struct cmap_node node; struct ct_dpif_timeout_policy policy; };