From patchwork Thu Jun 30 12:24:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valerio X-Patchwork-Id: 1650755 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=I//kop7T; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LYcwQ6sltz9s0r for ; Thu, 30 Jun 2022 22:24:54 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id B60D061366; Thu, 30 Jun 2022 12:24:52 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org B60D061366 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=I//kop7T X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SvstoqsSmanq; Thu, 30 Jun 2022 12:24:51 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp3.osuosl.org (Postfix) with ESMTPS id 7FCA76134E; Thu, 30 Jun 2022 12:24:50 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 7FCA76134E Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 46508C0021; Thu, 30 Jun 2022 12:24:50 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8297BC0011 for ; Thu, 30 Jun 2022 12:24:48 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 4B59984629 for ; Thu, 30 Jun 2022 12:24:48 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 4B59984629 Authentication-Results: smtp1.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=I//kop7T X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ac4qNgzNtX38 for ; Thu, 30 Jun 2022 12:24:47 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 322D581AEF Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id 322D581AEF for ; Thu, 30 Jun 2022 12:24:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1656591885; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iyjX1zla/MAKp4qeqrM9ZdmRrINc/MouYuroqpWHHmI=; b=I//kop7TcNTyjxKebb0f/+s52HCWzqLkGV/HLB+y5uYVchRw4O33C4zLK/NSQGgSzervOd 21wV4fvniTvHx6dQSTw/TsdVbSJADWT7A1W3zVQdjdQpripJ5W6mMA8BnrR5pOd06fM59w 89olkDiEtzc73cIcV2W+lJ8Ar/l8RA4= Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-659--WAq2m5uOd-z9db7kD1KsQ-1; Thu, 30 Jun 2022 08:24:45 -0400 X-MC-Unique: -WAq2m5uOd-z9db7kD1KsQ-1 Received: by mail-ed1-f70.google.com with SMTP id v16-20020a056402349000b00435a1c942a9so14309101edc.15 for ; Thu, 30 Jun 2022 05:24:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=iyjX1zla/MAKp4qeqrM9ZdmRrINc/MouYuroqpWHHmI=; b=lu9zTU2MldHTkjWErUMyoKf9pZIhPs6TFObdeyWIXrHw5yMZ5bICBOWXqvXxpXx/LI BtxnytoqZwvbaZndjlsuvGn+lN2CcgmfqcizPtqhsq6J8vYx+H8c/zRG9YhEroqGc/uq +uWCGl88dzDf1+1n4Nds7DhOilFrSSMfXj3C355RmfPaKLSER/PPswopYQetMKP4H6wg 11SgID2Hj4Hqi2XZbmJQFHbkGpOGnsap9gkhoT7wV3y+2nWT/T9h+DJHE4Qz2wdYluRr MyicDqR+KjMTkihuU5EUjEygUuTBpRS5eo2jqX/cBJf8wgZt01dvZEfX2XeKNEBRvKQB 4Vmw== X-Gm-Message-State: AJIora9nlNcqcsp+TkNnpWf5MQYisz7wHo5mvrlqR1Tma1kEIBQP8BWl yJMXQivdH3TQ1Kbgdhk1wKjgMR7e4ECqW48FbO/n4+GazTLVf7BlhL/GJTRqG2L4sWTWcGnASS8 LwYip/aBWkwfrsorME6GXG+XUd+MfM/woeDRipflTkjKRObju3LYoXLuH3WEVoXUG X-Received: by 2002:a17:907:1dde:b0:72a:4bc2:1625 with SMTP id og30-20020a1709071dde00b0072a4bc21625mr5565918ejc.144.1656591883410; Thu, 30 Jun 2022 05:24:43 -0700 (PDT) X-Google-Smtp-Source: AGRyM1tSP6ONQ5wsvS8CX1s5Jp7/cesRavbINd8M+M7ZsgAaCitx2CaHEnsFDkBwauHCuU5CTxG9Eg== X-Received: by 2002:a17:907:1dde:b0:72a:4bc2:1625 with SMTP id og30-20020a1709071dde00b0072a4bc21625mr5565878ejc.144.1656591882946; Thu, 30 Jun 2022 05:24:42 -0700 (PDT) Received: from localhost (net-5-95-131-226.cust.vodafonedsl.it. [5.95.131.226]) by smtp.gmail.com with ESMTPSA id r12-20020a170906c28c00b006ff52dfccf3sm9041820ejz.211.2022.06.30.05.24.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Jun 2022 05:24:42 -0700 (PDT) From: Paolo Valerio To: dev@openvswitch.org Date: Thu, 30 Jun 2022 14:24:41 +0200 Message-ID: <165659188184.1543933.1819403723887892312.stgit@fed.void> In-Reply-To: <165659186399.1543933.7694766166921029898.stgit@fed.void> References: <165659186399.1543933.7694766166921029898.stgit@fed.void> User-Agent: StGit/1.1 MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pvalerio@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: fbl@sysclose.org, i.maximets@ovn.org Subject: [ovs-dev] [PATCH v5 1/5] conntrack: Use a cmap to store zone limits X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Gaetan Rivet Change the data structure from hmap to cmap for zone limits. As they are shared amongst multiple conntrack users, multiple readers want to check the current zone limit state before progressing in their processing. Using a CMAP allows doing lookups without taking the global 'ct_lock', thus reducing contention. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Signed-off-by: Paolo Valerio --- lib/conntrack-private.h | 2 + lib/conntrack.c | 70 ++++++++++++++++++++++++++++++++--------------- lib/conntrack.h | 2 + lib/dpif-netdev.c | 5 ++- 4 files changed, 53 insertions(+), 26 deletions(-) diff --git a/lib/conntrack-private.h b/lib/conntrack-private.h index dfdf4e676..d9461b811 100644 --- a/lib/conntrack-private.h +++ b/lib/conntrack-private.h @@ -192,7 +192,7 @@ struct conntrack { struct ovs_mutex ct_lock; /* Protects 2 following fields. */ struct cmap conns OVS_GUARDED; struct ovs_list exp_lists[N_CT_TM] OVS_GUARDED; - struct hmap zone_limits OVS_GUARDED; + struct cmap zone_limits OVS_GUARDED; struct hmap timeout_policies OVS_GUARDED; uint32_t hash_basis; /* Salt for hashing a connection key. */ pthread_t clean_thread; /* Periodically cleans up connection tracker. */ diff --git a/lib/conntrack.c b/lib/conntrack.c index faa2d6ab7..6df1142b9 100644 --- a/lib/conntrack.c +++ b/lib/conntrack.c @@ -81,7 +81,7 @@ enum ct_alg_ctl_type { }; struct zone_limit { - struct hmap_node node; + struct cmap_node node; struct conntrack_zone_limit czl; }; @@ -311,7 +311,7 @@ conntrack_init(void) for (unsigned i = 0; i < ARRAY_SIZE(ct->exp_lists); i++) { ovs_list_init(&ct->exp_lists[i]); } - hmap_init(&ct->zone_limits); + cmap_init(&ct->zone_limits); ct->zone_limit_seq = 0; timeout_policy_init(ct); ovs_mutex_unlock(&ct->ct_lock); @@ -346,12 +346,25 @@ zone_key_hash(int32_t zone, uint32_t basis) } static struct zone_limit * -zone_limit_lookup(struct conntrack *ct, int32_t zone) +zone_limit_lookup_protected(struct conntrack *ct, int32_t zone) OVS_REQUIRES(ct->ct_lock) { uint32_t hash = zone_key_hash(zone, ct->hash_basis); struct zone_limit *zl; - HMAP_FOR_EACH_IN_BUCKET (zl, node, hash, &ct->zone_limits) { + CMAP_FOR_EACH_WITH_HASH_PROTECTED (zl, node, hash, &ct->zone_limits) { + if (zl->czl.zone == zone) { + return zl; + } + } + return NULL; +} + +static struct zone_limit * +zone_limit_lookup(struct conntrack *ct, int32_t zone) +{ + uint32_t hash = zone_key_hash(zone, ct->hash_basis); + struct zone_limit *zl; + CMAP_FOR_EACH_WITH_HASH (zl, node, hash, &ct->zone_limits) { if (zl->czl.zone == zone) { return zl; } @@ -361,7 +374,6 @@ zone_limit_lookup(struct conntrack *ct, int32_t zone) static struct zone_limit * zone_limit_lookup_or_default(struct conntrack *ct, int32_t zone) - OVS_REQUIRES(ct->ct_lock) { struct zone_limit *zl = zone_limit_lookup(ct, zone); return zl ? zl : zone_limit_lookup(ct, DEFAULT_ZONE); @@ -370,13 +382,16 @@ zone_limit_lookup_or_default(struct conntrack *ct, int32_t zone) struct conntrack_zone_limit zone_limit_get(struct conntrack *ct, int32_t zone) { - ovs_mutex_lock(&ct->ct_lock); - struct conntrack_zone_limit czl = {DEFAULT_ZONE, 0, 0, 0}; + struct conntrack_zone_limit czl = { + .zone = DEFAULT_ZONE, + .limit = 0, + .count = ATOMIC_COUNT_INIT(0), + .zone_limit_seq = 0, + }; struct zone_limit *zl = zone_limit_lookup_or_default(ct, zone); if (zl) { czl = zl->czl; } - ovs_mutex_unlock(&ct->ct_lock); return czl; } @@ -384,13 +399,19 @@ static int zone_limit_create(struct conntrack *ct, int32_t zone, uint32_t limit) OVS_REQUIRES(ct->ct_lock) { + struct zone_limit *zl = zone_limit_lookup_protected(ct, zone); + + if (zl) { + return 0; + } + if (zone >= DEFAULT_ZONE && zone <= MAX_ZONE) { - struct zone_limit *zl = xzalloc(sizeof *zl); + zl = xzalloc(sizeof *zl); zl->czl.limit = limit; zl->czl.zone = zone; zl->czl.zone_limit_seq = ct->zone_limit_seq++; uint32_t hash = zone_key_hash(zone, ct->hash_basis); - hmap_insert(&ct->zone_limits, &zl->node, hash); + cmap_insert(&ct->zone_limits, &zl->node, hash); return 0; } else { return EINVAL; @@ -401,13 +422,14 @@ int zone_limit_update(struct conntrack *ct, int32_t zone, uint32_t limit) { int err = 0; - ovs_mutex_lock(&ct->ct_lock); struct zone_limit *zl = zone_limit_lookup(ct, zone); if (zl) { zl->czl.limit = limit; VLOG_INFO("Changed zone limit of %u for zone %d", limit, zone); } else { + ovs_mutex_lock(&ct->ct_lock); err = zone_limit_create(ct, zone, limit); + ovs_mutex_unlock(&ct->ct_lock); if (!err) { VLOG_INFO("Created zone limit of %u for zone %d", limit, zone); } else { @@ -415,7 +437,6 @@ zone_limit_update(struct conntrack *ct, int32_t zone, uint32_t limit) zone); } } - ovs_mutex_unlock(&ct->ct_lock); return err; } @@ -423,23 +444,25 @@ static void zone_limit_clean(struct conntrack *ct, struct zone_limit *zl) OVS_REQUIRES(ct->ct_lock) { - hmap_remove(&ct->zone_limits, &zl->node); - free(zl); + uint32_t hash = zone_key_hash(zl->czl.zone, ct->hash_basis); + cmap_remove(&ct->zone_limits, &zl->node, hash); + ovsrcu_postpone(free, zl); } int zone_limit_delete(struct conntrack *ct, uint16_t zone) { ovs_mutex_lock(&ct->ct_lock); - struct zone_limit *zl = zone_limit_lookup(ct, zone); + struct zone_limit *zl = zone_limit_lookup_protected(ct, zone); if (zl) { zone_limit_clean(ct, zl); + ovs_mutex_unlock(&ct->ct_lock); VLOG_INFO("Deleted zone limit for zone %d", zone); } else { + ovs_mutex_unlock(&ct->ct_lock); VLOG_INFO("Attempted delete of non-existent zone limit: zone %d", zone); } - ovs_mutex_unlock(&ct->ct_lock); return 0; } @@ -456,7 +479,7 @@ conn_clean_cmn(struct conntrack *ct, struct conn *conn) struct zone_limit *zl = zone_limit_lookup(ct, conn->admit_zone); if (zl && zl->czl.zone_limit_seq == conn->zone_limit_seq) { - zl->czl.count--; + atomic_count_dec(&zl->czl.count); } } @@ -510,10 +533,13 @@ conntrack_destroy(struct conntrack *ct) cmap_destroy(&ct->conns); struct zone_limit *zl; - HMAP_FOR_EACH_POP (zl, node, &ct->zone_limits) { - free(zl); + CMAP_FOR_EACH (zl, node, &ct->zone_limits) { + uint32_t hash = zone_key_hash(zl->czl.zone, ct->hash_basis); + + cmap_remove(&ct->zone_limits, &zl->node, hash); + ovsrcu_postpone(free, zl); } - hmap_destroy(&ct->zone_limits); + cmap_destroy(&ct->zone_limits); struct timeout_policy *tp; HMAP_FOR_EACH_POP (tp, node, &ct->timeout_policies) { @@ -991,7 +1017,7 @@ conn_not_found(struct conntrack *ct, struct dp_packet *pkt, if (commit) { struct zone_limit *zl = zone_limit_lookup_or_default(ct, ctx->key.zone); - if (zl && zl->czl.count >= zl->czl.limit) { + if (zl && atomic_count_get(&zl->czl.count) >= zl->czl.limit) { return nc; } @@ -1064,7 +1090,7 @@ conn_not_found(struct conntrack *ct, struct dp_packet *pkt, if (zl) { nc->admit_zone = zl->czl.zone; nc->zone_limit_seq = zl->czl.zone_limit_seq; - zl->czl.count++; + atomic_count_inc(&zl->czl.count); } else { nc->admit_zone = INVALID_ZONE; } diff --git a/lib/conntrack.h b/lib/conntrack.h index 9553b188a..58b181834 100644 --- a/lib/conntrack.h +++ b/lib/conntrack.h @@ -108,7 +108,7 @@ struct conntrack_dump { struct conntrack_zone_limit { int32_t zone; uint32_t limit; - uint32_t count; + atomic_count count; uint32_t zone_limit_seq; /* Used to disambiguate zone limit counts. */ }; diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index d09138f2c..760cde1a4 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -9263,7 +9263,8 @@ dpif_netdev_ct_get_limits(struct dpif *dpif, czl = zone_limit_get(dp->conntrack, zone_limit->zone); if (czl.zone == zone_limit->zone || czl.zone == DEFAULT_ZONE) { ct_dpif_push_zone_limit(zone_limits_reply, zone_limit->zone, - czl.limit, czl.count); + czl.limit, + atomic_count_get(&czl.count)); } else { return EINVAL; } @@ -9273,7 +9274,7 @@ dpif_netdev_ct_get_limits(struct dpif *dpif, czl = zone_limit_get(dp->conntrack, z); if (czl.zone == z) { ct_dpif_push_zone_limit(zone_limits_reply, z, czl.limit, - czl.count); + atomic_count_get(&czl.count)); } } } From patchwork Thu Jun 30 12:24:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valerio X-Patchwork-Id: 1650756 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=VyyCUQqU; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LYcwX5lLrz9s0r for ; Thu, 30 Jun 2022 22:25:00 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id B5CCF6137A; Thu, 30 Jun 2022 12:24:58 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org B5CCF6137A Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=VyyCUQqU X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id U7Z1ZPCfaM85; Thu, 30 Jun 2022 12:24:57 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 5585361368; Thu, 30 Jun 2022 12:24:56 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 5585361368 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0B03CC0032; Thu, 30 Jun 2022 12:24:56 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 9918BC0011 for ; Thu, 30 Jun 2022 12:24:54 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 7407161371 for ; Thu, 30 Jun 2022 12:24:54 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 7407161371 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YkeGdd1FLriH for ; Thu, 30 Jun 2022 12:24:53 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 185C861367 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id 185C861367 for ; Thu, 30 Jun 2022 12:24:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1656591891; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DQ8AJ5QUxo9XnIkLf81k7Qz4hZ0b0lm4rQh44tOhYXk=; b=VyyCUQqU55GIhMRdNDt8oEp0oaWiXUiQPsJhd4lcdW9Mnc2hDIKHb/ze6ApVQ7CG7l0JP+ XHn8rfNaqtiFZ1GeulWSblYyUbXNAuQwdylgYPjxCBOt+MrBkV2HE7Cj0BXIKkrmqj9aQr elFQjOZoOb1nNzmfds7TMLzIKiwM4BY= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-407-0NHU49QOOp6shNBz9xYr0Q-1; Thu, 30 Jun 2022 08:24:51 -0400 X-MC-Unique: 0NHU49QOOp6shNBz9xYr0Q-1 Received: by mail-ed1-f72.google.com with SMTP id w8-20020a056402268800b004379267f163so9334947edd.20 for ; Thu, 30 Jun 2022 05:24:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=DQ8AJ5QUxo9XnIkLf81k7Qz4hZ0b0lm4rQh44tOhYXk=; b=n5ryTnNdS8obuE5K3uEWHFWbKpSyZ5NnFSO5yiyZmL7xI/6dfD3jXxvX2qC9a64nag wPg/TO/ms8zvDAExpzWGlpUQvscDQAD+RDQBV/n0BOBbqLglEu3M5KSS6HvaCcf9fVOF CeL9l4kGPiTMOkmHI6DRSxVYc/ByYiKau++XPQ+XH+ysLbxNbZknF9JSF5amqDUq0e1O i/IKQlL2mcJAKFfX3DIoy6YCrpk1dUPGBMRPhq/R7RvhvJ8YhskEP//4Qz5GP5An0YoW nwOtLWBS2Z1AyrjbPVAdoYV4J3WG6dx07X4SgkN2kRThnz5hVGad0Dhbfsp/C4Gcyi6k LiTQ== X-Gm-Message-State: AJIora8H7yewStkS09yEvLc3BI2h0UdPac5JOd7q9ZeTEXIAhj3an/Kv cGckjXiioFt+5BzSpgFGnEuqze6bSFnDaGVTotsXlwrz6/rMzHdINxwD7djsnf/ZpejzuOqOb7W na1yJUMDMPjKsM1cO6E+bbPqYWlP3GCXkcDetF5ZayosO3pIgjpMrjwwOw4M5OjqP X-Received: by 2002:a17:907:a05c:b0:72a:3959:43db with SMTP id gz28-20020a170907a05c00b0072a395943dbmr7400058ejc.359.1656591889515; Thu, 30 Jun 2022 05:24:49 -0700 (PDT) X-Google-Smtp-Source: AGRyM1tHwVX8XG0DGxh52UgsEnCONhormXoKZzQv6OpuMizlm01FHvymHgu509rXTgKQMt9VWT7o4w== X-Received: by 2002:a17:907:a05c:b0:72a:3959:43db with SMTP id gz28-20020a170907a05c00b0072a395943dbmr7400034ejc.359.1656591889200; Thu, 30 Jun 2022 05:24:49 -0700 (PDT) Received: from localhost (net-5-95-131-226.cust.vodafonedsl.it. [5.95.131.226]) by smtp.gmail.com with ESMTPSA id n4-20020aa7c784000000b00437e08d319csm1904594eds.61.2022.06.30.05.24.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Jun 2022 05:24:48 -0700 (PDT) From: Paolo Valerio To: dev@openvswitch.org Date: Thu, 30 Jun 2022 14:24:48 +0200 Message-ID: <165659188809.1543933.13937807799328406589.stgit@fed.void> In-Reply-To: <165659186399.1543933.7694766166921029898.stgit@fed.void> References: <165659186399.1543933.7694766166921029898.stgit@fed.void> User-Agent: StGit/1.1 MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pvalerio@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: fbl@sysclose.org, i.maximets@ovn.org Subject: [ovs-dev] [PATCH v5 2/5] conntrack-tp: Use a cmap to store timeout policies X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Gaetan Rivet Multiple lookups are done to stored timeout policies, each time blocking the global 'ct_lock'. This is usually not necessary and it should be acceptable to get policy updates slightly delayed (by one RCU sync at most). Using a CMAP reduces multiple lock taking and releasing in the connection insertion path. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Acked-by: William Tu Signed-off-by: Paolo Valerio --- lib/conntrack-private.h | 2 +- lib/conntrack-tp.c | 54 ++++++++++++++++++++++++++--------------------- lib/conntrack.c | 9 +++++--- lib/conntrack.h | 2 +- 4 files changed, 38 insertions(+), 29 deletions(-) diff --git a/lib/conntrack-private.h b/lib/conntrack-private.h index d9461b811..34c688821 100644 --- a/lib/conntrack-private.h +++ b/lib/conntrack-private.h @@ -193,7 +193,7 @@ struct conntrack { struct cmap conns OVS_GUARDED; struct ovs_list exp_lists[N_CT_TM] OVS_GUARDED; struct cmap zone_limits OVS_GUARDED; - struct hmap timeout_policies OVS_GUARDED; + struct cmap timeout_policies OVS_GUARDED; uint32_t hash_basis; /* Salt for hashing a connection key. */ pthread_t clean_thread; /* Periodically cleans up connection tracker. */ struct latch clean_thread_exit; /* To destroy the 'clean_thread'. */ diff --git a/lib/conntrack-tp.c b/lib/conntrack-tp.c index a586d3a8d..c2245038b 100644 --- a/lib/conntrack-tp.c +++ b/lib/conntrack-tp.c @@ -47,14 +47,15 @@ static unsigned int ct_dpif_netdev_tp_def[] = { }; static struct timeout_policy * -timeout_policy_lookup(struct conntrack *ct, int32_t tp_id) +timeout_policy_lookup_protected(struct conntrack *ct, int32_t tp_id) OVS_REQUIRES(ct->ct_lock) { struct timeout_policy *tp; uint32_t hash; hash = hash_int(tp_id, ct->hash_basis); - HMAP_FOR_EACH_IN_BUCKET (tp, node, hash, &ct->timeout_policies) { + CMAP_FOR_EACH_WITH_HASH_PROTECTED (tp, node, hash, + &ct->timeout_policies) { if (tp->policy.id == tp_id) { return tp; } @@ -62,20 +63,25 @@ timeout_policy_lookup(struct conntrack *ct, int32_t tp_id) return NULL; } -struct timeout_policy * -timeout_policy_get(struct conntrack *ct, int32_t tp_id) +static struct timeout_policy * +timeout_policy_lookup(struct conntrack *ct, int32_t tp_id) { struct timeout_policy *tp; + uint32_t hash; - ovs_mutex_lock(&ct->ct_lock); - tp = timeout_policy_lookup(ct, tp_id); - if (!tp) { - ovs_mutex_unlock(&ct->ct_lock); - return NULL; + hash = hash_int(tp_id, ct->hash_basis); + CMAP_FOR_EACH_WITH_HASH (tp, node, hash, &ct->timeout_policies) { + if (tp->policy.id == tp_id) { + return tp; + } } + return NULL; +} - ovs_mutex_unlock(&ct->ct_lock); - return tp; +struct timeout_policy * +timeout_policy_get(struct conntrack *ct, int32_t tp_id) +{ + return timeout_policy_lookup(ct, tp_id); } static void @@ -125,27 +131,30 @@ timeout_policy_create(struct conntrack *ct, init_default_tp(tp, tp_id); update_existing_tp(tp, new_tp); hash = hash_int(tp_id, ct->hash_basis); - hmap_insert(&ct->timeout_policies, &tp->node, hash); + cmap_insert(&ct->timeout_policies, &tp->node, hash); } static void timeout_policy_clean(struct conntrack *ct, struct timeout_policy *tp) OVS_REQUIRES(ct->ct_lock) { - hmap_remove(&ct->timeout_policies, &tp->node); - free(tp); + uint32_t hash = hash_int(tp->policy.id, ct->hash_basis); + cmap_remove(&ct->timeout_policies, &tp->node, hash); + ovsrcu_postpone(free, tp); } static int -timeout_policy_delete__(struct conntrack *ct, uint32_t tp_id) +timeout_policy_delete__(struct conntrack *ct, uint32_t tp_id, + bool warn_on_error) OVS_REQUIRES(ct->ct_lock) { + struct timeout_policy *tp; int err = 0; - struct timeout_policy *tp = timeout_policy_lookup(ct, tp_id); + tp = timeout_policy_lookup_protected(ct, tp_id); if (tp) { timeout_policy_clean(ct, tp); - } else { + } else if (warn_on_error) { VLOG_WARN_RL(&rl, "Failed to delete a non-existent timeout " "policy: id=%d", tp_id); err = ENOENT; @@ -159,7 +168,7 @@ timeout_policy_delete(struct conntrack *ct, uint32_t tp_id) int err; ovs_mutex_lock(&ct->ct_lock); - err = timeout_policy_delete__(ct, tp_id); + err = timeout_policy_delete__(ct, tp_id, true); ovs_mutex_unlock(&ct->ct_lock); return err; } @@ -170,7 +179,7 @@ timeout_policy_init(struct conntrack *ct) { struct timeout_policy tp; - hmap_init(&ct->timeout_policies); + cmap_init(&ct->timeout_policies); /* Create default timeout policy. */ memset(&tp, 0, sizeof tp); @@ -182,14 +191,11 @@ int timeout_policy_update(struct conntrack *ct, struct timeout_policy *new_tp) { - int err = 0; uint32_t tp_id = new_tp->policy.id; + int err = 0; ovs_mutex_lock(&ct->ct_lock); - struct timeout_policy *tp = timeout_policy_lookup(ct, tp_id); - if (tp) { - err = timeout_policy_delete__(ct, tp_id); - } + timeout_policy_delete__(ct, tp_id, false); timeout_policy_create(ct, new_tp); ovs_mutex_unlock(&ct->ct_lock); return err; diff --git a/lib/conntrack.c b/lib/conntrack.c index 6df1142b9..38ddc9b91 100644 --- a/lib/conntrack.c +++ b/lib/conntrack.c @@ -542,10 +542,13 @@ conntrack_destroy(struct conntrack *ct) cmap_destroy(&ct->zone_limits); struct timeout_policy *tp; - HMAP_FOR_EACH_POP (tp, node, &ct->timeout_policies) { - free(tp); + CMAP_FOR_EACH (tp, node, &ct->timeout_policies) { + uint32_t hash = hash_int(tp->policy.id, ct->hash_basis); + + cmap_remove(&ct->timeout_policies, &tp->node, hash); + ovsrcu_postpone(free, tp); } - hmap_destroy(&ct->timeout_policies); + cmap_destroy(&ct->timeout_policies); ovs_mutex_unlock(&ct->ct_lock); ovs_mutex_destroy(&ct->ct_lock); diff --git a/lib/conntrack.h b/lib/conntrack.h index 58b181834..b064abc9f 100644 --- a/lib/conntrack.h +++ b/lib/conntrack.h @@ -113,7 +113,7 @@ struct conntrack_zone_limit { }; struct timeout_policy { - struct hmap_node node; + struct cmap_node node; struct ct_dpif_timeout_policy policy; }; From patchwork Thu Jun 30 12:24:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valerio X-Patchwork-Id: 1650757 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=MToaD1Tb; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LYcww0NQwz9s0r for ; Thu, 30 Jun 2022 22:25:19 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 007188457C; Thu, 30 Jun 2022 12:25:18 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 007188457C Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=MToaD1Tb X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3Z4TUH3aiUfJ; Thu, 30 Jun 2022 12:25:16 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 8B30084643; Thu, 30 Jun 2022 12:25:15 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 8B30084643 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 4B28AC0021; Thu, 30 Jun 2022 12:25:15 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 6A7E6C0011 for ; Thu, 30 Jun 2022 12:25:14 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id D6D0961365 for ; Thu, 30 Jun 2022 12:25:01 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org D6D0961365 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=MToaD1Tb X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7jADyNs3zVSz for ; Thu, 30 Jun 2022 12:25:00 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org E89A461379 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id E89A461379 for ; Thu, 30 Jun 2022 12:24:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1656591898; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zKJ3kNmMkt32AgCfwtzi5NSWnjFpk6rjbY5zlwSUWMs=; b=MToaD1TbcD5lpq3Pw52jSoYnsf1oADZVVvQRq681ceiDOxGNsIQJF3d5CO5m/8fB06VnnN VyKZVTYjXeCSHpA2fqAyKYiHEGwipBPEmpON9lu54ZWmjmhTBOyY6ewBkCY+hIqSTczqrV 8TZafzuXDZZ63n9iroYPzsxxaC6N+0Y= Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-379-uUSf8iN1MvCPZwy7sYcl5Q-1; Thu, 30 Jun 2022 08:24:57 -0400 X-MC-Unique: uUSf8iN1MvCPZwy7sYcl5Q-1 Received: by mail-ed1-f69.google.com with SMTP id m8-20020a056402430800b00435cfa7c6d1so14186015edc.9 for ; Thu, 30 Jun 2022 05:24:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=hq++Xd3YJL9WNkX9j226yj1/J/WjCctz1Ed1LINofxg=; b=H0T25NH0hp9SSQeRYVJwpR71+NuefjjwnKvm9keUz6UCIl2X998XfCd5vLeIT9aYyE ZsaP5oFsMm6nbDZ2pwJbZ2Pv8lDTUbSUm2e8u92CU+E8XEpK9FQQfY+kVsYNEJEYsUDQ RYMLvCjGNGg6Mq0zQXTZlRXUNp1lxzWbHw0JKpjsYlU1Y/gpRvd956+lTLxeRwKu2H0w xYhXvjkWsGpZ5z1IqW9yLiqKbP1GgvQl7K9Nm2OwzmHyF8rI3tcrR55s1v6v8bXNWRgC rwURRqisACMBlVnrvsr++OZgyECYEgkRQGu0WHznJWS9U90HDZF6r5xZF+59AYw2LAjf lsjQ== X-Gm-Message-State: AJIora/9mM0/cnBFfopDEzpPWO98Xj7lkxpRhTQ0HCZoWKhtHGfJXj9F 1AqCKxFiQX3M+X4e67GPHB4iJ+8850uM03pUJgP/4OaGCvCRkaBWkXwk/X8y/jRbeLIHhWEsj/0 FfzCE2bvGlQ828bmSzl1AGPwS7fwqBuiRi3NDxII+Vn7OYdR3V5+56zwQP+m6QRxI X-Received: by 2002:a17:906:cc87:b0:722:fb3e:9f9c with SMTP id oq7-20020a170906cc8700b00722fb3e9f9cmr8516517ejb.624.1656591896032; Thu, 30 Jun 2022 05:24:56 -0700 (PDT) X-Google-Smtp-Source: AGRyM1stz2wwvjbFjmDKRv0m6gfM07M8BwKtCAygLGT2qYaydCd8ccag6XmNkzvJ9RbUQPzKKRHqpA== X-Received: by 2002:a17:906:cc87:b0:722:fb3e:9f9c with SMTP id oq7-20020a170906cc8700b00722fb3e9f9cmr8516477ejb.624.1656591895397; Thu, 30 Jun 2022 05:24:55 -0700 (PDT) Received: from localhost (net-5-95-131-226.cust.vodafonedsl.it. [5.95.131.226]) by smtp.gmail.com with ESMTPSA id k2-20020a17090666c200b00715a02874acsm8886371ejp.35.2022.06.30.05.24.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Jun 2022 05:24:55 -0700 (PDT) From: Paolo Valerio To: dev@openvswitch.org Date: Thu, 30 Jun 2022 14:24:54 +0200 Message-ID: <165659189431.1543933.3871386028207934269.stgit@fed.void> In-Reply-To: <165659186399.1543933.7694766166921029898.stgit@fed.void> References: <165659186399.1543933.7694766166921029898.stgit@fed.void> User-Agent: StGit/1.1 MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pvalerio@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: fbl@sysclose.org, i.maximets@ovn.org Subject: [ovs-dev] [PATCH v5 3/5] conntrack: Replace timeout based expiration lists with rculists. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Gaetan Rivet This patch aims to replace the expiration lists as, due to the way they are used, besides being a source of contention, they have a known issue when used with non-default policies for different zones that could lead to retaining expired connections potentially for a long time. This patch replaces them with an array of rculist used to distribute all the newly created connections in order to, during the sweeping phase, scan them without locking, and evict the expired connections only locking during the actual removal. This allows to reduce the contention introduced by the pushback performed at every packet update, also solving the issue that affects the current implementation. Signed-off-by: Gaetan Rivet Co-authored-by: Paolo Valerio Signed-off-by: Paolo Valerio --- - I kept the authorship as, although this changes the approach used in [0] some code got reused. - The number of lists could be changed if too big, but should not be reduced to a very small number. - The worst case latency for the nth entry is (sweeper-reschedule-time + time-to-scan-up-to-the-nth-element). This latency is the same as the bucket series [1]. [0] https://patchwork.ozlabs.org/project/openvswitch/patch/9bb167046210836960f0957c5c11c76019b18f1b.1648231764.git.grive@u256.net/ [1] https://patchwork.ozlabs.org/project/openvswitch/patch/164781566120.306037.12928694623127213139.stgit@fed.void/ --- lib/conntrack-private.h | 84 ++++++++++++++++++--------- lib/conntrack-tp.c | 45 +------------- lib/conntrack.c | 146 ++++++++++++++++++++++++----------------------- 3 files changed, 132 insertions(+), 143 deletions(-) diff --git a/lib/conntrack-private.h b/lib/conntrack-private.h index 34c688821..676f58d83 100644 --- a/lib/conntrack-private.h +++ b/lib/conntrack-private.h @@ -29,6 +29,7 @@ #include "openvswitch/list.h" #include "openvswitch/types.h" #include "packets.h" +#include "rculist.h" #include "unaligned.h" #include "dp-packet.h" @@ -86,6 +87,31 @@ struct alg_exp_node { bool nat_rpl_dst; }; +/* Timeouts: all the possible timeout states passed to update_expiration() + * are listed here. The name will be prefix by CT_TM_ and the value is in + * milliseconds */ +#define CT_TIMEOUTS \ + CT_TIMEOUT(TCP_FIRST_PACKET) \ + CT_TIMEOUT(TCP_OPENING) \ + CT_TIMEOUT(TCP_ESTABLISHED) \ + CT_TIMEOUT(TCP_CLOSING) \ + CT_TIMEOUT(TCP_FIN_WAIT) \ + CT_TIMEOUT(TCP_CLOSED) \ + CT_TIMEOUT(OTHER_FIRST) \ + CT_TIMEOUT(OTHER_MULTIPLE) \ + CT_TIMEOUT(OTHER_BIDIR) \ + CT_TIMEOUT(ICMP_FIRST) \ + CT_TIMEOUT(ICMP_REPLY) + +enum ct_timeout { +#define CT_TIMEOUT(NAME) CT_TM_##NAME, + CT_TIMEOUTS +#undef CT_TIMEOUT + N_CT_TM +}; + +#define EXP_LISTS 100 + enum OVS_PACKED_ENUM ct_conn_type { CT_CONN_TYPE_DEFAULT, CT_CONN_TYPE_UN_NAT, @@ -96,11 +122,16 @@ struct conn { struct conn_key key; struct conn_key rev_key; struct conn_key parent_key; /* Only used for orig_tuple support. */ - struct ovs_list exp_node; struct cmap_node cm_node; uint16_t nat_action; char *alg; struct conn *nat_conn; /* The NAT 'conn' context, if there is one. */ + atomic_flag reclaimed; /* False during the lifetime of the connection, + * True as soon as a thread has started freeing + * its memory. */ + + /* Inserted once by a PMD, then managed by the 'ct_clean' thread. */ + struct rculist node; /* Mutable data. */ struct ovs_mutex lock; /* Guards all mutable fields. */ @@ -116,7 +147,6 @@ struct conn { /* Mutable data. */ bool seq_skew_dir; /* TCP sequence skew direction due to NATTing of FTP * control messages; true if reply direction. */ - bool cleaned; /* True if cleaned from expiry lists. */ /* Immutable data. */ bool alg_related; /* True if alg data connection. */ @@ -132,22 +162,6 @@ enum ct_update_res { CT_UPDATE_VALID_NEW, }; -/* Timeouts: all the possible timeout states passed to update_expiration() - * are listed here. The name will be prefix by CT_TM_ and the value is in - * milliseconds */ -#define CT_TIMEOUTS \ - CT_TIMEOUT(TCP_FIRST_PACKET) \ - CT_TIMEOUT(TCP_OPENING) \ - CT_TIMEOUT(TCP_ESTABLISHED) \ - CT_TIMEOUT(TCP_CLOSING) \ - CT_TIMEOUT(TCP_FIN_WAIT) \ - CT_TIMEOUT(TCP_CLOSED) \ - CT_TIMEOUT(OTHER_FIRST) \ - CT_TIMEOUT(OTHER_MULTIPLE) \ - CT_TIMEOUT(OTHER_BIDIR) \ - CT_TIMEOUT(ICMP_FIRST) \ - CT_TIMEOUT(ICMP_REPLY) - #define NAT_ACTION_SNAT_ALL (NAT_ACTION_SRC | NAT_ACTION_SRC_PORT) #define NAT_ACTION_DNAT_ALL (NAT_ACTION_DST | NAT_ACTION_DST_PORT) @@ -181,22 +195,17 @@ enum ct_ephemeral_range { #define FOR_EACH_PORT_IN_RANGE(curr, min, max) \ FOR_EACH_PORT_IN_RANGE__(curr, min, max, OVS_JOIN(idx, __COUNTER__)) -enum ct_timeout { -#define CT_TIMEOUT(NAME) CT_TM_##NAME, - CT_TIMEOUTS -#undef CT_TIMEOUT - N_CT_TM -}; - struct conntrack { struct ovs_mutex ct_lock; /* Protects 2 following fields. */ struct cmap conns OVS_GUARDED; - struct ovs_list exp_lists[N_CT_TM] OVS_GUARDED; + struct rculist exp_lists[EXP_LISTS]; struct cmap zone_limits OVS_GUARDED; struct cmap timeout_policies OVS_GUARDED; uint32_t hash_basis; /* Salt for hashing a connection key. */ pthread_t clean_thread; /* Periodically cleans up connection tracker. */ struct latch clean_thread_exit; /* To destroy the 'clean_thread'. */ + atomic_uint ct_next_list; + unsigned int next_sweep; /* Counting connections. */ atomic_count n_conn; /* Number of connections currently tracked. */ @@ -216,8 +225,8 @@ struct conntrack { }; /* Lock acquisition order: - * 1. 'ct_lock' - * 2. 'conn->lock' + * 1. 'conn->lock' + * 2. 'ct_lock' * 3. 'resources_lock' */ @@ -237,4 +246,23 @@ struct ct_l4_proto { struct ct_dpif_protoinfo *); }; +static unsigned int +ct_next_list(struct conntrack *ct) +{ + unsigned int old; + + atomic_add_relaxed(&ct->ct_next_list, 1u, &old); + + return old % EXP_LISTS; +} + +static inline void +conn_expire_push_front(struct conntrack *ct, struct conn *conn) + OVS_REQUIRES(ct->ct_lock) +{ + unsigned int next = ct_next_list(ct); + + rculist_push_front(&ct->exp_lists[next], &conn->node); +} + #endif /* conntrack-private.h */ diff --git a/lib/conntrack-tp.c b/lib/conntrack-tp.c index c2245038b..8d17af00a 100644 --- a/lib/conntrack-tp.c +++ b/lib/conntrack-tp.c @@ -236,71 +236,30 @@ tm_to_ct_dpif_tp(enum ct_timeout tm) return CT_DPIF_TP_ATTR_MAX; } -static void -conn_update_expiration__(struct conntrack *ct, struct conn *conn, - enum ct_timeout tm, long long now, - uint32_t tp_value) - OVS_REQUIRES(conn->lock) -{ - ovs_mutex_unlock(&conn->lock); - - ovs_mutex_lock(&ct->ct_lock); - ovs_mutex_lock(&conn->lock); - if (!conn->cleaned) { - conn->expiration = now + tp_value * 1000; - ovs_list_remove(&conn->exp_node); - ovs_list_push_back(&ct->exp_lists[tm], &conn->exp_node); - } - ovs_mutex_unlock(&conn->lock); - ovs_mutex_unlock(&ct->ct_lock); - - ovs_mutex_lock(&conn->lock); -} - /* The conn entry lock must be held on entry and exit. */ void conn_update_expiration(struct conntrack *ct, struct conn *conn, enum ct_timeout tm, long long now) - OVS_REQUIRES(conn->lock) { struct timeout_policy *tp; uint32_t val; - ovs_mutex_unlock(&conn->lock); - - ovs_mutex_lock(&ct->ct_lock); - ovs_mutex_lock(&conn->lock); tp = timeout_policy_lookup(ct, conn->tp_id); if (tp) { val = tp->policy.attrs[tm_to_ct_dpif_tp(tm)]; } else { val = ct_dpif_netdev_tp_def[tm_to_ct_dpif_tp(tm)]; } - ovs_mutex_unlock(&conn->lock); - ovs_mutex_unlock(&ct->ct_lock); - - ovs_mutex_lock(&conn->lock); VLOG_DBG_RL(&rl, "Update timeout %s zone=%u with policy id=%d " "val=%u sec.", ct_timeout_str[tm], conn->key.zone, conn->tp_id, val); - conn_update_expiration__(ct, conn, tm, now, val); + conn->expiration = now + val * 1000; } -static void -conn_init_expiration__(struct conntrack *ct, struct conn *conn, - enum ct_timeout tm, long long now, - uint32_t tp_value) -{ - conn->expiration = now + tp_value * 1000; - ovs_list_push_back(&ct->exp_lists[tm], &conn->exp_node); -} - -/* ct_lock must be held. */ void conn_init_expiration(struct conntrack *ct, struct conn *conn, enum ct_timeout tm, long long now) - OVS_REQUIRES(ct->ct_lock) { struct timeout_policy *tp; uint32_t val; @@ -315,5 +274,5 @@ conn_init_expiration(struct conntrack *ct, struct conn *conn, VLOG_DBG_RL(&rl, "Init timeout %s zone=%u with policy id=%d val=%u sec.", ct_timeout_str[tm], conn->key.zone, conn->tp_id, val); - conn_init_expiration__(ct, conn, tm, now, val); + conn->expiration = now + val * 1000; } diff --git a/lib/conntrack.c b/lib/conntrack.c index 38ddc9b91..ffe1d6ada 100644 --- a/lib/conntrack.c +++ b/lib/conntrack.c @@ -39,12 +39,12 @@ #include "ovs-thread.h" #include "openvswitch/poll-loop.h" #include "random.h" +#include "rculist.h" #include "timeval.h" VLOG_DEFINE_THIS_MODULE(conntrack); COVERAGE_DEFINE(conntrack_full); -COVERAGE_DEFINE(conntrack_long_cleanup); COVERAGE_DEFINE(conntrack_l3csum_err); COVERAGE_DEFINE(conntrack_l4csum_err); COVERAGE_DEFINE(conntrack_lookup_natted_miss); @@ -96,7 +96,6 @@ static struct conn *new_conn(struct conntrack *ct, struct dp_packet *pkt, uint32_t tp_id); static void delete_conn_cmn(struct conn *); static void delete_conn(struct conn *); -static void delete_conn_one(struct conn *conn); static enum ct_update_res conn_update(struct conntrack *ct, struct conn *conn, struct dp_packet *pkt, struct conn_lookup_ctx *ctx, @@ -309,7 +308,7 @@ conntrack_init(void) ovs_mutex_lock(&ct->ct_lock); cmap_init(&ct->conns); for (unsigned i = 0; i < ARRAY_SIZE(ct->exp_lists); i++) { - ovs_list_init(&ct->exp_lists[i]); + rculist_init(&ct->exp_lists[i]); } cmap_init(&ct->zone_limits); ct->zone_limit_seq = 0; @@ -319,6 +318,7 @@ conntrack_init(void) atomic_count_init(&ct->n_conn, 0); atomic_init(&ct->n_conn_limit, DEFAULT_N_CONN_LIMIT); atomic_init(&ct->tcp_seq_chk, true); + atomic_init(&ct->ct_next_list, 0); latch_init(&ct->clean_thread_exit); ct->clean_thread = ovs_thread_create("ct_clean", clean_thread_main, ct); ct->ipf = ipf_init(); @@ -468,7 +468,7 @@ zone_limit_delete(struct conntrack *ct, uint16_t zone) static void conn_clean_cmn(struct conntrack *ct, struct conn *conn) - OVS_REQUIRES(ct->ct_lock) + OVS_REQUIRES(conn->lock, ct->ct_lock) { if (conn->alg) { expectation_clean(ct, &conn->key); @@ -487,32 +487,38 @@ conn_clean_cmn(struct conntrack *ct, struct conn *conn) * removes the associated nat 'conn' from the lookup datastructures. */ static void conn_clean(struct conntrack *ct, struct conn *conn) - OVS_REQUIRES(ct->ct_lock) + OVS_EXCLUDED(conn->lock, ct->ct_lock) { ovs_assert(conn->conn_type == CT_CONN_TYPE_DEFAULT); + if (atomic_flag_test_and_set(&conn->reclaimed)) { + return; + } + + ovs_mutex_lock(&conn->lock); + + ovs_mutex_lock(&ct->ct_lock); conn_clean_cmn(ct, conn); if (conn->nat_conn) { uint32_t hash = conn_key_hash(&conn->nat_conn->key, ct->hash_basis); cmap_remove(&ct->conns, &conn->nat_conn->cm_node, hash); } - ovs_list_remove(&conn->exp_node); - conn->cleaned = true; + + rculist_remove(&conn->node); + ovs_mutex_unlock(&ct->ct_lock); + ovsrcu_postpone(delete_conn, conn); atomic_count_dec(&ct->n_conn); + + ovs_mutex_unlock(&conn->lock); } static void -conn_clean_one(struct conntrack *ct, struct conn *conn) - OVS_REQUIRES(ct->ct_lock) +conn_force_expire(struct conn *conn) { - conn_clean_cmn(ct, conn); - if (conn->conn_type == CT_CONN_TYPE_DEFAULT) { - ovs_list_remove(&conn->exp_node); - conn->cleaned = true; - atomic_count_dec(&ct->n_conn); - } - ovsrcu_postpone(delete_conn_one, conn); + ovs_mutex_lock(&conn->lock); + conn->expiration = 0; + ovs_mutex_unlock(&conn->lock); } /* Destroys the connection tracker 'ct' and frees all the allocated memory. @@ -522,15 +528,16 @@ void conntrack_destroy(struct conntrack *ct) { struct conn *conn; + latch_set(&ct->clean_thread_exit); pthread_join(ct->clean_thread, NULL); latch_destroy(&ct->clean_thread_exit); - ovs_mutex_lock(&ct->ct_lock); - CMAP_FOR_EACH (conn, cm_node, &ct->conns) { - conn_clean_one(ct, conn); + for (unsigned i = 0; i < EXP_LISTS; i++) { + RCULIST_FOR_EACH (conn, node, &ct->exp_lists[i]) { + conn_clean(ct, conn); + } } - cmap_destroy(&ct->conns); struct zone_limit *zl; CMAP_FOR_EACH (zl, node, &ct->zone_limits) { @@ -539,7 +546,6 @@ conntrack_destroy(struct conntrack *ct) cmap_remove(&ct->zone_limits, &zl->node, hash); ovsrcu_postpone(free, zl); } - cmap_destroy(&ct->zone_limits); struct timeout_policy *tp; CMAP_FOR_EACH (tp, node, &ct->timeout_policies) { @@ -548,6 +554,11 @@ conntrack_destroy(struct conntrack *ct) cmap_remove(&ct->timeout_policies, &tp->node, hash); ovsrcu_postpone(free, tp); } + + ovs_mutex_lock(&ct->ct_lock); + + cmap_destroy(&ct->conns); + cmap_destroy(&ct->zone_limits); cmap_destroy(&ct->timeout_policies); ovs_mutex_unlock(&ct->ct_lock); @@ -1087,7 +1098,9 @@ conn_not_found(struct conntrack *ct, struct dp_packet *pkt, nc->nat_conn = nat_conn; ovs_mutex_init_adaptive(&nc->lock); nc->conn_type = CT_CONN_TYPE_DEFAULT; + atomic_flag_clear(&nc->reclaimed); cmap_insert(&ct->conns, &nc->cm_node, ctx->hash); + conn_expire_push_front(ct, nc); atomic_count_inc(&ct->n_conn); ctx->conn = nc; /* For completeness. */ if (zl) { @@ -1108,7 +1121,6 @@ conn_not_found(struct conntrack *ct, struct dp_packet *pkt, * can limit DoS impact. */ nat_res_exhaustion: free(nat_conn); - ovs_list_remove(&nc->exp_node); delete_conn_cmn(nc); static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(5, 5); VLOG_WARN_RL(&rl, "Unable to NAT due to tuple space exhaustion - " @@ -1148,11 +1160,9 @@ conn_update_state(struct conntrack *ct, struct dp_packet *pkt, pkt->md.ct_state = CS_INVALID; break; case CT_UPDATE_NEW: - ovs_mutex_lock(&ct->ct_lock); if (conn_lookup(ct, &conn->key, now, NULL, NULL)) { - conn_clean(ct, conn); + conn_force_expire(conn); } - ovs_mutex_unlock(&ct->ct_lock); create_new_conn = true; break; case CT_UPDATE_VALID_NEW: @@ -1363,11 +1373,9 @@ process_one(struct conntrack *ct, struct dp_packet *pkt, /* Delete found entry if in wrong direction. 'force' implies commit. */ if (OVS_UNLIKELY(force && ctx->reply && conn)) { - ovs_mutex_lock(&ct->ct_lock); if (conn_lookup(ct, &conn->key, now, NULL, NULL)) { - conn_clean(ct, conn); + conn_force_expire(conn); } - ovs_mutex_unlock(&ct->ct_lock); conn = NULL; } @@ -1553,39 +1561,21 @@ set_label(struct dp_packet *pkt, struct conn *conn, * LLONG_MAX if 'ctb' is empty. The return value might be smaller than 'now', * if 'limit' is reached */ static long long -ct_sweep(struct conntrack *ct, long long now, size_t limit) +ct_sweep(struct conntrack *ct, struct rculist *list, long long now) + OVS_NO_THREAD_SAFETY_ANALYSIS { struct conn *conn; - long long min_expiration = LLONG_MAX; size_t count = 0; - ovs_mutex_lock(&ct->ct_lock); - - for (unsigned i = 0; i < N_CT_TM; i++) { - LIST_FOR_EACH_SAFE (conn, exp_node, &ct->exp_lists[i]) { - ovs_mutex_lock(&conn->lock); - if (now < conn->expiration || count >= limit) { - min_expiration = MIN(min_expiration, conn->expiration); - ovs_mutex_unlock(&conn->lock); - if (count >= limit) { - /* Do not check other lists. */ - COVERAGE_INC(conntrack_long_cleanup); - goto out; - } - break; - } else { - ovs_mutex_unlock(&conn->lock); - conn_clean(ct, conn); - } - count++; + RCULIST_FOR_EACH (conn, node, list) { + if (conn_expired(conn, now)) { + conn_clean(ct, conn); } + + count++; } -out: - VLOG_DBG("conntrack cleanup %"PRIuSIZE" entries in %lld msec", count, - time_msec() - now); - ovs_mutex_unlock(&ct->ct_lock); - return min_expiration; + return count; } /* Cleans up old connection entries from 'ct'. Returns the time when the @@ -1595,11 +1585,26 @@ out: static long long conntrack_clean(struct conntrack *ct, long long now) { - unsigned int n_conn_limit; + long long next_wakeup = now + 30 * 1000; + unsigned int n_conn_limit, i, count = 0; + size_t clean_end; + atomic_read_relaxed(&ct->n_conn_limit, &n_conn_limit); - size_t clean_max = n_conn_limit > 10 ? n_conn_limit / 10 : 1; - long long min_exp = ct_sweep(ct, now, clean_max); - long long next_wakeup = MIN(min_exp, now + CT_DPIF_NETDEV_TP_MIN); + clean_end = n_conn_limit / 64; + + for (i = ct->next_sweep; i < EXP_LISTS; i++) { + count += ct_sweep(ct, &ct->exp_lists[i], now); + + if (count > clean_end) { + next_wakeup = 0; + break; + } + } + + ct->next_sweep = (i < EXP_LISTS) ? i : 0; + + VLOG_DBG("conntrack cleanup %"PRIu32" entries in %lld msec", count, + time_msec() - now); return next_wakeup; } @@ -1628,6 +1633,7 @@ conntrack_clean(struct conntrack *ct, long long now) static void * clean_thread_main(void *f_) + OVS_NO_THREAD_SAFETY_ANALYSIS { struct conntrack *ct = f_; @@ -2554,15 +2560,6 @@ delete_conn(struct conn *conn) delete_conn_cmn(conn); } -/* Only used by conn_clean_one(). */ -static void -delete_conn_one(struct conn *conn) -{ - if (conn->conn_type == CT_CONN_TYPE_DEFAULT) { - ovs_mutex_destroy(&conn->lock); - } - delete_conn_cmn(conn); -} /* Convert a conntrack address 'a' into an IP address 'b' based on 'dl_type'. * @@ -2714,6 +2711,11 @@ conntrack_dump_next(struct conntrack_dump *dump, struct ct_dpif_entry *entry) } struct conn *conn; INIT_CONTAINER(conn, cm_node, cm_node); + + if (conn_expired(conn, now)) { + continue; + } + if ((!dump->filter_zone || conn->key.zone == dump->zone) && (conn->conn_type != CT_CONN_TYPE_UN_NAT)) { conn_to_ct_dpif_entry(conn, entry, now); @@ -2735,13 +2737,15 @@ conntrack_flush(struct conntrack *ct, const uint16_t *zone) { struct conn *conn; - ovs_mutex_lock(&ct->ct_lock); CMAP_FOR_EACH (conn, cm_node, &ct->conns) { + if (conn->conn_type != CT_CONN_TYPE_DEFAULT) { + continue; + } + if (!zone || *zone == conn->key.zone) { - conn_clean_one(ct, conn); + conn_clean(ct, conn); } } - ovs_mutex_unlock(&ct->ct_lock); return 0; } @@ -2756,7 +2760,6 @@ conntrack_flush_tuple(struct conntrack *ct, const struct ct_dpif_tuple *tuple, memset(&key, 0, sizeof(key)); tuple_to_conn_key(tuple, zone, &key); - ovs_mutex_lock(&ct->ct_lock); conn_lookup(ct, &key, time_msec(), &conn, NULL); if (conn && conn->conn_type == CT_CONN_TYPE_DEFAULT) { @@ -2766,7 +2769,6 @@ conntrack_flush_tuple(struct conntrack *ct, const struct ct_dpif_tuple *tuple, error = ENOENT; } - ovs_mutex_unlock(&ct->ct_lock); return error; } From patchwork Thu Jun 30 12:25:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valerio X-Patchwork-Id: 1650759 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=UGtsOFIM; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LYcxP2rwxz9s0r for ; Thu, 30 Jun 2022 22:25:45 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id A2760415C3; Thu, 30 Jun 2022 12:25:43 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org A2760415C3 Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=UGtsOFIM X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ja8U-s6XIiiv; Thu, 30 Jun 2022 12:25:42 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp2.osuosl.org (Postfix) with ESMTPS id AF5E9415BE; Thu, 30 Jun 2022 12:25:41 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org AF5E9415BE Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 677A2C0032; Thu, 30 Jun 2022 12:25:41 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 36E79C0021 for ; Thu, 30 Jun 2022 12:25:40 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 5CED3415BE for ; Thu, 30 Jun 2022 12:25:06 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 5CED3415BE X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XTa-qOh2Kmw4 for ; Thu, 30 Jun 2022 12:25:05 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 6A34C4119A Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id 6A34C4119A for ; Thu, 30 Jun 2022 12:25:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1656591904; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sq0duvNZOcH4OC8ETgo1M7alNXz4cBYL+9Tx+E/igRo=; b=UGtsOFIMcRzCXEfBqcrOBUZ+2MhlsDSufyKmgR+eUWlR9Y9JjS5JBPul4VbZFxMXxzfv2b Bd6EKI+haAY/x9GNF+Ci5pqoPBjT2/G+fRtY7K/hFIpjCkFezUKH1PSy9aduvQFOjZNuda GE6NxOOgVuJIKafH8Itj7XlzgMYZWJo= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-386-PTxmr1aEPjy5hRSsw3YKVg-1; Thu, 30 Jun 2022 08:25:03 -0400 X-MC-Unique: PTxmr1aEPjy5hRSsw3YKVg-1 Received: by mail-ed1-f72.google.com with SMTP id w8-20020a056402268800b004379267f163so9335331edd.20 for ; Thu, 30 Jun 2022 05:25:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=sq0duvNZOcH4OC8ETgo1M7alNXz4cBYL+9Tx+E/igRo=; b=whBVbeW4xUtO0uLnVtyJduoJOUR+3LKxOvqNgcVB7gaMyLbDauYG4cx5EUJ3CBM0bZ +BSSY/pkCHoBD3ddxo1wkaLAgJIuAGM1kx7J34s9eDrWMP7fREeDDGYk+dVSj3F1njzf PiOkO5Zc5+noUJuctRfjxGkp7DUiZBwwYMywsMNYZdwgmd2eBLwmfMvggxljSygI0xxU YeaGew2+7BJ1o3HshpgSgYxYGQZtnLLqg1kMOGOfUELu0bqPdcu2jUiaErEwMX1zRtWG 2qwbZkaQGFECkFTM650W9DJZcelWwhOfv9swsjtiXzjoSfYO7PSl222PgZK/YyXrtgJ+ 5r5g== X-Gm-Message-State: AJIora8bnSU36c8Ojf109Mt/qa0YpoD1DKbCisf56rl+4Xhwp6n4ddNW 7UwJJjytkQtcfyBM5MQWQjvyG4TY9YkhDCaXfNNd1Kq75shh6kfmaxI4wiopD8UiHD2/jdaQzQm ot5kIok+NB9+u/yL7iw7HcA14o+nRi5fC6/i28dqoWJ1iTwDVcHD7pR6mwwugoUz7 X-Received: by 2002:aa7:cb01:0:b0:437:8a8a:dbb3 with SMTP id s1-20020aa7cb01000000b004378a8adbb3mr10982343edt.74.1656591901849; Thu, 30 Jun 2022 05:25:01 -0700 (PDT) X-Google-Smtp-Source: AGRyM1vnkbh95ML/q1RoPCqdyR1hVwJrvADnSjU6K3Y0yRRrlbqjUWXI6f/ZpktMjU0MYN/jANVudQ== X-Received: by 2002:aa7:cb01:0:b0:437:8a8a:dbb3 with SMTP id s1-20020aa7cb01000000b004378a8adbb3mr10982320edt.74.1656591901587; Thu, 30 Jun 2022 05:25:01 -0700 (PDT) Received: from localhost (net-5-95-131-226.cust.vodafonedsl.it. [5.95.131.226]) by smtp.gmail.com with ESMTPSA id a21-20020a056402169500b004357063bf60sm13075798edv.41.2022.06.30.05.25.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Jun 2022 05:25:01 -0700 (PDT) From: Paolo Valerio To: dev@openvswitch.org Date: Thu, 30 Jun 2022 14:25:00 +0200 Message-ID: <165659190053.1543933.6459408442479101161.stgit@fed.void> In-Reply-To: <165659186399.1543933.7694766166921029898.stgit@fed.void> References: <165659186399.1543933.7694766166921029898.stgit@fed.void> User-Agent: StGit/1.1 MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pvalerio@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: fbl@sysclose.org, i.maximets@ovn.org Subject: [ovs-dev] [PATCH v5 4/5] conntrack: Use an atomic conn expiration value X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Gaetan Rivet A lock is taken during conn_lookup() to check whether a connection is expired before returning it. This lock can have some contention. Even though this lock ensures a consistent sequence of writes, it does not imply a specific order. A ct_clean thread taking the lock first could read a value that would be updated immediately after by a PMD waiting on the same lock, just as well as the inverse order. As such, the expiration time can be stale anytime it is read. In this context, using an atomic will ensure the same guarantees for either writes or reads, i.e. writes are consistent and reads are not undefined behaviour. Reading an atomic is however less costly than taking and releasing a lock. Signed-off-by: Gaetan Rivet Reviewed-by: Eli Britstein Acked-by: William Tu Signed-off-by: Paolo Valerio --- lib/conntrack-private.h | 2 +- lib/conntrack.c | 23 ++++++++++++++--------- 2 files changed, 15 insertions(+), 10 deletions(-) diff --git a/lib/conntrack-private.h b/lib/conntrack-private.h index 676f58d83..d36845c2d 100644 --- a/lib/conntrack-private.h +++ b/lib/conntrack-private.h @@ -136,7 +136,7 @@ struct conn { /* Mutable data. */ struct ovs_mutex lock; /* Guards all mutable fields. */ ovs_u128 label; - long long expiration; + atomic_llong expiration; uint32_t mark; int seq_skew; diff --git a/lib/conntrack.c b/lib/conntrack.c index ffe1d6ada..057653781 100644 --- a/lib/conntrack.c +++ b/lib/conntrack.c @@ -100,6 +100,7 @@ static enum ct_update_res conn_update(struct conntrack *ct, struct conn *conn, struct dp_packet *pkt, struct conn_lookup_ctx *ctx, long long now); +static long long int conn_expiration(const struct conn *); static bool conn_expired(struct conn *, long long now); static void set_mark(struct dp_packet *, struct conn *, uint32_t val, uint32_t mask); @@ -973,13 +974,10 @@ un_nat_packet(struct dp_packet *pkt, const struct conn *conn, static void conn_seq_skew_set(struct conntrack *ct, const struct conn *conn_in, long long now, int seq_skew, bool seq_skew_dir) - OVS_NO_THREAD_SAFETY_ANALYSIS { struct conn *conn; - ovs_mutex_unlock(&conn_in->lock); - conn_lookup(ct, &conn_in->key, now, &conn, NULL); - ovs_mutex_lock(&conn_in->lock); + conn_lookup(ct, &conn_in->key, now, &conn, NULL); if (conn && seq_skew) { conn->seq_skew = seq_skew; conn->seq_skew_dir = seq_skew_dir; @@ -2519,14 +2517,21 @@ conn_update(struct conntrack *ct, struct conn *conn, struct dp_packet *pkt, return update_res; } +static long long int +conn_expiration(const struct conn *conn) +{ + long long int expiration; + + atomic_read_relaxed(&CONST_CAST(struct conn *, conn)->expiration, + &expiration); + return expiration; +} + static bool conn_expired(struct conn *conn, long long now) { if (conn->conn_type == CT_CONN_TYPE_DEFAULT) { - ovs_mutex_lock(&conn->lock); - bool expired = now >= conn->expiration ? true : false; - ovs_mutex_unlock(&conn->lock); - return expired; + return now >= conn_expiration(conn); } return false; } @@ -2659,7 +2664,7 @@ conn_to_ct_dpif_entry(const struct conn *conn, struct ct_dpif_entry *entry, entry->mark = conn->mark; memcpy(&entry->labels, &conn->label, sizeof entry->labels); - long long expiration = conn->expiration - now; + long long expiration = conn_expiration(conn) - now; struct ct_l4_proto *class = l4_protos[conn->key.nw_proto]; if (class->conn_get_protoinfo) { From patchwork Thu Jun 30 12:25:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valerio X-Patchwork-Id: 1650758 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=ekFv8lV4; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4LYcxB26svz9s0r for ; Thu, 30 Jun 2022 22:25:33 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 04761415A4; Thu, 30 Jun 2022 12:25:32 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 04761415A4 Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=ekFv8lV4 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qRSRK2CEVd2y; Thu, 30 Jun 2022 12:25:31 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 38E214119A; Thu, 30 Jun 2022 12:25:30 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 38E214119A Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 19B89C0032; Thu, 30 Jun 2022 12:25:30 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0E5CBC0021 for ; Thu, 30 Jun 2022 12:25:29 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 8D6DD423BE for ; Thu, 30 Jun 2022 12:25:15 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 8D6DD423BE Authentication-Results: smtp4.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=ekFv8lV4 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4dwUT4532Ie8 for ; Thu, 30 Jun 2022 12:25:14 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 83011423CE Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id 83011423CE for ; Thu, 30 Jun 2022 12:25:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1656591913; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NYChUsWF0cI9YSCRhYi/r3fizoXVwauI4Y0fpPVkS1E=; b=ekFv8lV4Y0SeBzTcfxYyI5FD0KuIRoUFnnTSp/GM+mXVgVc8w5SeKJbVuVE26/hDWNezpK mFpCmwgMkgOo+1tRicIvRJJ5fDnKT/ulx/XsapOkx8iwGeLaEpo6B2mSY4RTJr1TzUa0LS Qv5LZHyYuQYto0y+dhqHPCE30KmFNsU= Received: from mail-ej1-f72.google.com (mail-ej1-f72.google.com [209.85.218.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-154-lgcZODZ9NwadRGKsbjKdtw-1; Thu, 30 Jun 2022 08:25:09 -0400 X-MC-Unique: lgcZODZ9NwadRGKsbjKdtw-1 Received: by mail-ej1-f72.google.com with SMTP id qa41-20020a17090786a900b00722f313a60eso6178822ejc.13 for ; Thu, 30 Jun 2022 05:25:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=NYChUsWF0cI9YSCRhYi/r3fizoXVwauI4Y0fpPVkS1E=; b=2yDX2E/grfs5pfz58B4+Gcs+Gk/ksSZId1ZFgmVsNftZb1wl5byPjYdCMoJsyBZOTS n46AynGMjSEBO4Q+4SIrm5FOLmqMeuhYrDaG2RjOYHSnzzOHsZg1tt/0FKzlsiTEpP+8 1vBFTCBmA86hc4lHH8VTWtrhWuQeQduiSbfWOZ4dp1Nt1VVk2mVSxkZKjWNv702JjghO rxAestLLWGEyhX67DVU63gvrBPEj3nWDHRqLE6+6AfzYFzcSHveqVYlnjGZwsCFDnpJo zaoZqGCXVo8ZoyptcNsmyN00waDEP9fRc76cymDDK/fKmaI0J7wNYdOU8bfyT3nh+VES fbpw== X-Gm-Message-State: AJIora/vgODe1Px0o0iROzOyFiOUOkcI1OMjDAyvj/NxETUn2XckjZ9L 4tJNdHFEhuOgf4nFT8Z4+FG6o31XjyT4XlyMsOxS0vemY3ztgjje5lJxPP4ppVlyBU7Rnd2k7Pu OaS4pcnb8109SzMRK7C+HIwP2k24w2oUZ9Rs5GtxjSu686cRG2qvF3yuw7CA6pp66 X-Received: by 2002:a17:906:6d91:b0:715:7d4e:84a2 with SMTP id h17-20020a1709066d9100b007157d4e84a2mr8628657ejt.504.1656591908053; Thu, 30 Jun 2022 05:25:08 -0700 (PDT) X-Google-Smtp-Source: AGRyM1sp70hgA7dNHPNtcXaGNwpNLNlnJnZrCnGusmVzCBevXEkx/UvKvLGxbRJHF5YqSKoQ0rtO2Q== X-Received: by 2002:a17:906:6d91:b0:715:7d4e:84a2 with SMTP id h17-20020a1709066d9100b007157d4e84a2mr8628622ejt.504.1656591907778; Thu, 30 Jun 2022 05:25:07 -0700 (PDT) Received: from localhost (net-5-95-131-226.cust.vodafonedsl.it. [5.95.131.226]) by smtp.gmail.com with ESMTPSA id dx13-20020a170906a84d00b0070f7d1c5a18sm9024621ejb.55.2022.06.30.05.25.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Jun 2022 05:25:07 -0700 (PDT) From: Paolo Valerio To: dev@openvswitch.org Date: Thu, 30 Jun 2022 14:25:06 +0200 Message-ID: <165659190668.1543933.15119029109781169279.stgit@fed.void> In-Reply-To: <165659186399.1543933.7694766166921029898.stgit@fed.void> References: <165659186399.1543933.7694766166921029898.stgit@fed.void> User-Agent: StGit/1.1 MIME-Version: 1.0 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pvalerio@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: fbl@sysclose.org, i.maximets@ovn.org Subject: [ovs-dev] [PATCH v5 5/5] conntrack: Check for expiration before comparing the keys during the lookup X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" From: Ilya Maximets This could save some costly key comparison miss, especially in the case there are many expired connections waiting for the sweeper to evict them. Signed-off-by: Ilya Maximets Signed-off-by: Paolo Valerio --- lib/conntrack.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/lib/conntrack.c b/lib/conntrack.c index 057653781..a2b8f5562 100644 --- a/lib/conntrack.c +++ b/lib/conntrack.c @@ -589,14 +589,17 @@ conn_key_lookup(struct conntrack *ct, const struct conn_key *key, bool found = false; CMAP_FOR_EACH_WITH_HASH (conn, cm_node, hash, &ct->conns) { - if (!conn_key_cmp(&conn->key, key) && !conn_expired(conn, now)) { + if (conn_expired(conn, now)) { + continue; + } + if (!conn_key_cmp(&conn->key, key)) { found = true; if (reply) { *reply = false; } break; } - if (!conn_key_cmp(&conn->rev_key, key) && !conn_expired(conn, now)) { + if (!conn_key_cmp(&conn->rev_key, key)) { found = true; if (reply) { *reply = true;