From patchwork Fri Jul 1 18:14:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valerio X-Patchwork-Id: 1651342 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Hmivv7aE; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LZNdB5qdPz9ryY for ; Sat, 2 Jul 2022 04:14:22 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id EEC0C8470B; Fri, 1 Jul 2022 18:14:20 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org EEC0C8470B Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Hmivv7aE X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TphFl4lpGDH1; Fri, 1 Jul 2022 18:14:19 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id A193B8474A; Fri, 1 Jul 2022 18:14:18 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org A193B8474A Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 6D3F3C0032; Fri, 1 Jul 2022 18:14:18 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 27090C0036 for ; Fri, 1 Jul 2022 18:14:17 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 8C35D424D1 for ; Fri, 1 Jul 2022 18:14:13 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 8C35D424D1 Authentication-Results: smtp4.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Hmivv7aE X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hW6BQF5LOhu8 for ; Fri, 1 Jul 2022 18:14:11 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 62E77424C7 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id 62E77424C7 for ; Fri, 1 Jul 2022 18:14:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1656699250; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DQ8AJ5QUxo9XnIkLf81k7Qz4hZ0b0lm4rQh44tOhYXk=; b=Hmivv7aEQJh+bs/P3XZZAn15pV6qhAnsfFKwV/vb7sM9AwPCi6UC70hKTlwzyE8vkjY1HB 83ZF8RgcAEAdSAMlfzMgmC13zsYOK8K6yPahpbDCPGvOcuw2363zK5OFv3isL32WwrOMr/ bIwcH6aa4EjsMFO7t7djTy9P33WgEXM= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-213-ziQ7MSYdNt63UV8-QhONMw-1; Fri, 01 Jul 2022 14:14:09 -0400 X-MC-Unique: ziQ7MSYdNt63UV8-QhONMw-1 Received: by mail-wm1-f69.google.com with SMTP id 130-20020a1c0288000000b003a18127d11aso312562wmc.6 for ; Fri, 01 Jul 2022 11:14:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=DQ8AJ5QUxo9XnIkLf81k7Qz4hZ0b0lm4rQh44tOhYXk=; b=KCueHNOXRmgIs3zHfjqP07RWhjjn4pkA9dPQZdiyZiei12e6KW8gFsdLE8IAXgAWbE YtauriOS5gHjqf71MIXoeRQZhPQkkOsfgyvXHgDsfvucEuW8uK7yPw9nCIzVW3ZTuP3G jDK0AtVzPDbqIKYTkQOrVaD61WGRwnh3fhlXscKdMphNN2wHCkJeC2X/VfmPcZpUFApE xuV/lqgKCj7rUn90thpGa1f8jo4qUkAD96cM+2vphsc+Sz147gWwOvARoLbhW2TeBWbI b1iEf1pCGiHNiDrfvHH96111xy0qmvXJnqbD5woKCKsiW/7rBQzAD3aI+nQJKLsv1kYD GOaw== X-Gm-Message-State: AJIora+MIpvL5ytgCPJm3NPnkfeN4DbwNLk9zLvvq7KCTYlXx2MXuDlt YjhAfyGZZGoHB12diUNxr70hh4FCx6ySCuGochvzvyiIBCHfhIrpGViY4f8X/WwVWZ2uP/OeQoi 04LfIkTD5nSLpZbu2Rd1JoHJITSnhy482MG2VmcEZwCSDZIz+8po8KAEbNHn+NhaP X-Received: by 2002:a5d:62c4:0:b0:21b:c031:a94b with SMTP id o4-20020a5d62c4000000b0021bc031a94bmr14949925wrv.624.1656699247617; Fri, 01 Jul 2022 11:14:07 -0700 (PDT) X-Google-Smtp-Source: AGRyM1uwuqlfUzFUn1SucCtpKsqSQfs8lHlV7k8Py936P8KOhBZ8dQPpJ1zATJPBQZyk4/Mgxdycqw== X-Received: by 2002:a5d:62c4:0:b0:21b:c031:a94b with SMTP id o4-20020a5d62c4000000b0021bc031a94bmr14949901wrv.624.1656699247258; Fri, 01 Jul 2022 11:14:07 -0700 (PDT) Received: from localhost (net-5-95-131-226.cust.vodafonedsl.it. [5.95.131.226]) by smtp.gmail.com with ESMTPSA id n7-20020a5d67c7000000b0021b9e8d4c22sm23048340wrw.61.2022.07.01.11.14.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Jul 2022 11:14:06 -0700 (PDT) From: Paolo Valerio To: dev@openvswitch.org Date: Fri, 01 Jul 2022 20:14:06 +0200 Message-ID: <165669924610.2104762.3230988027335443233.stgit@fed.void> In-Reply-To: <165669922758.2104762.4398770071010254560.stgit@fed.void> References: <165669922758.2104762.4398770071010254560.stgit@fed.void> User-Agent: StGit/1.1 MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pvalerio@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: fbl@sysclose.org, i.maximets@ovn.org Subject: [ovs-dev] [PATCH v6 2/5] conntrack-tp: Use a cmap to store timeout policies X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Gaetan Rivet Multiple lookups are done to stored timeout policies, each time blocking the global 'ct_lock'. This is usually not necessary and it should be acceptable to get policy updates slightly delayed (by one RCU sync at most). Using a CMAP reduces multiple lock taking and releasing in the connection insertion path. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Acked-by: William Tu Signed-off-by: Paolo Valerio --- lib/conntrack-private.h | 2 +- lib/conntrack-tp.c | 54 ++++++++++++++++++++++++++--------------------- lib/conntrack.c | 9 +++++--- lib/conntrack.h | 2 +- 4 files changed, 38 insertions(+), 29 deletions(-) diff --git a/lib/conntrack-private.h b/lib/conntrack-private.h index d9461b811..34c688821 100644 --- a/lib/conntrack-private.h +++ b/lib/conntrack-private.h @@ -193,7 +193,7 @@ struct conntrack { struct cmap conns OVS_GUARDED; struct ovs_list exp_lists[N_CT_TM] OVS_GUARDED; struct cmap zone_limits OVS_GUARDED; - struct hmap timeout_policies OVS_GUARDED; + struct cmap timeout_policies OVS_GUARDED; uint32_t hash_basis; /* Salt for hashing a connection key. */ pthread_t clean_thread; /* Periodically cleans up connection tracker. */ struct latch clean_thread_exit; /* To destroy the 'clean_thread'. */ diff --git a/lib/conntrack-tp.c b/lib/conntrack-tp.c index a586d3a8d..c2245038b 100644 --- a/lib/conntrack-tp.c +++ b/lib/conntrack-tp.c @@ -47,14 +47,15 @@ static unsigned int ct_dpif_netdev_tp_def[] = { }; static struct timeout_policy * -timeout_policy_lookup(struct conntrack *ct, int32_t tp_id) +timeout_policy_lookup_protected(struct conntrack *ct, int32_t tp_id) OVS_REQUIRES(ct->ct_lock) { struct timeout_policy *tp; uint32_t hash; hash = hash_int(tp_id, ct->hash_basis); - HMAP_FOR_EACH_IN_BUCKET (tp, node, hash, &ct->timeout_policies) { + CMAP_FOR_EACH_WITH_HASH_PROTECTED (tp, node, hash, + &ct->timeout_policies) { if (tp->policy.id == tp_id) { return tp; } @@ -62,20 +63,25 @@ timeout_policy_lookup(struct conntrack *ct, int32_t tp_id) return NULL; } -struct timeout_policy * -timeout_policy_get(struct conntrack *ct, int32_t tp_id) +static struct timeout_policy * +timeout_policy_lookup(struct conntrack *ct, int32_t tp_id) { struct timeout_policy *tp; + uint32_t hash; - ovs_mutex_lock(&ct->ct_lock); - tp = timeout_policy_lookup(ct, tp_id); - if (!tp) { - ovs_mutex_unlock(&ct->ct_lock); - return NULL; + hash = hash_int(tp_id, ct->hash_basis); + CMAP_FOR_EACH_WITH_HASH (tp, node, hash, &ct->timeout_policies) { + if (tp->policy.id == tp_id) { + return tp; + } } + return NULL; +} - ovs_mutex_unlock(&ct->ct_lock); - return tp; +struct timeout_policy * +timeout_policy_get(struct conntrack *ct, int32_t tp_id) +{ + return timeout_policy_lookup(ct, tp_id); } static void @@ -125,27 +131,30 @@ timeout_policy_create(struct conntrack *ct, init_default_tp(tp, tp_id); update_existing_tp(tp, new_tp); hash = hash_int(tp_id, ct->hash_basis); - hmap_insert(&ct->timeout_policies, &tp->node, hash); + cmap_insert(&ct->timeout_policies, &tp->node, hash); } static void timeout_policy_clean(struct conntrack *ct, struct timeout_policy *tp) OVS_REQUIRES(ct->ct_lock) { - hmap_remove(&ct->timeout_policies, &tp->node); - free(tp); + uint32_t hash = hash_int(tp->policy.id, ct->hash_basis); + cmap_remove(&ct->timeout_policies, &tp->node, hash); + ovsrcu_postpone(free, tp); } static int -timeout_policy_delete__(struct conntrack *ct, uint32_t tp_id) +timeout_policy_delete__(struct conntrack *ct, uint32_t tp_id, + bool warn_on_error) OVS_REQUIRES(ct->ct_lock) { + struct timeout_policy *tp; int err = 0; - struct timeout_policy *tp = timeout_policy_lookup(ct, tp_id); + tp = timeout_policy_lookup_protected(ct, tp_id); if (tp) { timeout_policy_clean(ct, tp); - } else { + } else if (warn_on_error) { VLOG_WARN_RL(&rl, "Failed to delete a non-existent timeout " "policy: id=%d", tp_id); err = ENOENT; @@ -159,7 +168,7 @@ timeout_policy_delete(struct conntrack *ct, uint32_t tp_id) int err; ovs_mutex_lock(&ct->ct_lock); - err = timeout_policy_delete__(ct, tp_id); + err = timeout_policy_delete__(ct, tp_id, true); ovs_mutex_unlock(&ct->ct_lock); return err; } @@ -170,7 +179,7 @@ timeout_policy_init(struct conntrack *ct) { struct timeout_policy tp; - hmap_init(&ct->timeout_policies); + cmap_init(&ct->timeout_policies); /* Create default timeout policy. */ memset(&tp, 0, sizeof tp); @@ -182,14 +191,11 @@ int timeout_policy_update(struct conntrack *ct, struct timeout_policy *new_tp) { - int err = 0; uint32_t tp_id = new_tp->policy.id; + int err = 0; ovs_mutex_lock(&ct->ct_lock); - struct timeout_policy *tp = timeout_policy_lookup(ct, tp_id); - if (tp) { - err = timeout_policy_delete__(ct, tp_id); - } + timeout_policy_delete__(ct, tp_id, false); timeout_policy_create(ct, new_tp); ovs_mutex_unlock(&ct->ct_lock); return err; diff --git a/lib/conntrack.c b/lib/conntrack.c index 6df1142b9..38ddc9b91 100644 --- a/lib/conntrack.c +++ b/lib/conntrack.c @@ -542,10 +542,13 @@ conntrack_destroy(struct conntrack *ct) cmap_destroy(&ct->zone_limits); struct timeout_policy *tp; - HMAP_FOR_EACH_POP (tp, node, &ct->timeout_policies) { - free(tp); + CMAP_FOR_EACH (tp, node, &ct->timeout_policies) { + uint32_t hash = hash_int(tp->policy.id, ct->hash_basis); + + cmap_remove(&ct->timeout_policies, &tp->node, hash); + ovsrcu_postpone(free, tp); } - hmap_destroy(&ct->timeout_policies); + cmap_destroy(&ct->timeout_policies); ovs_mutex_unlock(&ct->ct_lock); ovs_mutex_destroy(&ct->ct_lock); diff --git a/lib/conntrack.h b/lib/conntrack.h index 58b181834..b064abc9f 100644 --- a/lib/conntrack.h +++ b/lib/conntrack.h @@ -113,7 +113,7 @@ struct conntrack_zone_limit { }; struct timeout_policy { - struct hmap_node node; + struct cmap_node node; struct ct_dpif_timeout_policy policy; };