From patchwork Mon Sep 30 20:50:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valerio X-Patchwork-Id: 1991193 X-Patchwork-Delegate: aconole@redhat.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=YOAvlQWY; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::137; helo=smtp4.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XHYC00wHmz1xtc for ; Tue, 1 Oct 2024 06:51:22 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id AB90B402BA; Mon, 30 Sep 2024 20:51:19 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id wRzQOv9yYGXL; Mon, 30 Sep 2024 20:51:18 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=2605:bc80:3010:104::8cd3:938; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org B05F3402BB Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=YOAvlQWY Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id B05F3402BB; Mon, 30 Sep 2024 20:51:18 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D3D0DC002B; Mon, 30 Sep 2024 20:51:17 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 58C65C002A for ; Mon, 30 Sep 2024 20:51:16 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 317D98104F for ; Mon, 30 Sep 2024 20:51:16 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id sHkiksR8Vi_B for ; Mon, 30 Sep 2024 20:51:15 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.129.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=pvalerio@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp1.osuosl.org 6A2388102F Authentication-Results: smtp1.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 6A2388102F Authentication-Results: smtp1.osuosl.org; dkim=pass (1024-bit key, unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=YOAvlQWY Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id 6A2388102F for ; Mon, 30 Sep 2024 20:51:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1727729473; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=xvp2+m/JA/N8SsrrFX0f7Bn2rLiofX/QSP8GWmxp+fA=; b=YOAvlQWYTv0oJ29uQzOlu5OtWHVcJICBKI7c5VBV31vrrnCsNpz1OAAXjND/+vZTWMfn85 GWTP6/Pz9NcE2yBPzhWW4FkMdHjorLWbjAmQF2NVwKejg30zggbEypiJiRVnWZXl/p5KZY 0dWzGf5EjlEWmzyOw33vRDIjr9qxP2U= Received: from mail-ej1-f71.google.com (mail-ej1-f71.google.com [209.85.218.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-79-AAn6geHkMsKyNkwcZlpL0w-1; Mon, 30 Sep 2024 16:51:12 -0400 X-MC-Unique: AAn6geHkMsKyNkwcZlpL0w-1 Received: by mail-ej1-f71.google.com with SMTP id a640c23a62f3a-a8a8d9a2a12so379001366b.3 for ; Mon, 30 Sep 2024 13:51:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727729470; x=1728334270; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=xvp2+m/JA/N8SsrrFX0f7Bn2rLiofX/QSP8GWmxp+fA=; b=LmXOJA7K2/a+OzuWYVKGwEmhZixtGsQXQerR5LtBbm6zoLq2HLUq+jDbZV/D63kdlN uGhsifmmLC4kJQCHMP/UZ6Lab9eIyIeMfPm8zKQamBc0RLR9WUIBfPenB33vJ+CWMYIl HEJN1uHRUXbS2K3Ie9mKQva/uK0/WI5Qa7JjqAdjPP5gxhvH/Fv2Qwxl17ezoRUDBNNs EnfM0EVltDvu9qIee9kXf92zx6rrOPB7WuL+3/nV7fSMyCa9LQHsAB1zq1ffW+Mb9lsS HBNfZIwPBfHu+CcxNGeuO94jZoQf6otY+9CUuQeVEYSO7EuJBK3h4+kRYqFR03gxIDW6 wH6Q== X-Gm-Message-State: AOJu0YzYNrDfT8alkfGU78qv2QNXmn4K+fA9ZLaj57mLvHLZFPHmPnz1 f83q20T2Jb68EaMdpw2VxkZm2YpssM6e5cWMs7TFkHoBbS0k640+kUWIWw+kqxsWHUhlGkOqfjS MCuZBNIsfVyck5vdngYYj8C4UOZzoJgheMOet5ez6KE9WEm5KR2ftSU8SxavQU1C6MtZ3FOIvzG ugtqXnq/yQnoP+Nfr99RdLO+XLHYRC8bW/QH3e/Gk= X-Received: by 2002:a17:907:7207:b0:a8d:2b7a:ff44 with SMTP id a640c23a62f3a-a93c492a6aamr1599330666b.32.1727729470508; Mon, 30 Sep 2024 13:51:10 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFuhC10b5qBbrs9f038/i5gp/MCfpK2S0GlIoXtmjsmbRS3wnBPJLXk5/gl5UaUvVAoD6wYMw== X-Received: by 2002:a17:907:7207:b0:a8d:2b7a:ff44 with SMTP id a640c23a62f3a-a93c492a6aamr1599328966b.32.1727729470125; Mon, 30 Sep 2024 13:51:10 -0700 (PDT) Received: from localhost (net-188-216-80-179.cust.vodafonedsl.it. [188.216.80.179]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a93c2996631sm579238666b.200.2024.09.30.13.51.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2024 13:51:09 -0700 (PDT) From: Paolo Valerio To: ovs-dev@openvswitch.org Date: Mon, 30 Sep 2024 22:50:29 +0200 Message-ID: <20240930205034.65484-1-pvalerio@redhat.com> X-Mailer: git-send-email 2.46.1 MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v2 1/6] conntrack: Correctly annotate conntrack member. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" While at it update no longer valid comment. Signed-off-by: Paolo Valerio Acked-by: Aaron Conole --- lib/conntrack-private.h | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/lib/conntrack-private.h b/lib/conntrack-private.h index 71367f211..6c65caa07 100644 --- a/lib/conntrack-private.h +++ b/lib/conntrack-private.h @@ -199,11 +199,12 @@ enum ct_ephemeral_range { FOR_EACH_PORT_IN_RANGE__(curr, min, max, OVS_JOIN(idx, __COUNTER__)) struct conntrack { - struct ovs_mutex ct_lock; /* Protects 2 following fields. */ + struct ovs_mutex ct_lock; /* Protects the following fields. */ struct cmap conns[UINT16_MAX + 1] OVS_GUARDED; - struct rculist exp_lists[N_EXP_LISTS]; + struct rculist exp_lists[N_EXP_LISTS] OVS_GUARDED; struct cmap zone_limits OVS_GUARDED; struct cmap timeout_policies OVS_GUARDED; + uint32_t hash_basis; /* Salt for hashing a connection key. */ pthread_t clean_thread; /* Periodically cleans up connection tracker. */ struct latch clean_thread_exit; /* To destroy the 'clean_thread'. */ From patchwork Mon Sep 30 20:50:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valerio X-Patchwork-Id: 1991194 X-Patchwork-Delegate: aconole@redhat.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=FUk1GzpO; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XHYC06Hwmz1xt8 for ; Tue, 1 Oct 2024 06:51:24 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 0658E408C0; Mon, 30 Sep 2024 20:51:22 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 9iNu0P7HLeAS; Mon, 30 Sep 2024 20:51:21 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org EF316401CC Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=FUk1GzpO Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id EF316401CC; Mon, 30 Sep 2024 20:51:20 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id DC549C0893; Mon, 30 Sep 2024 20:51:20 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 261D9C002B for ; Mon, 30 Sep 2024 20:51:19 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 14636810C9 for ; Mon, 30 Sep 2024 20:51:19 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 3ObqpSdL_fkH for ; Mon, 30 Sep 2024 20:51:18 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.129.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=pvalerio@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp1.osuosl.org 181858102F Authentication-Results: smtp1.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 181858102F Authentication-Results: smtp1.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=FUk1GzpO Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id 181858102F for ; Mon, 30 Sep 2024 20:51:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1727729476; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=osKeptO4NFC9rVaQiTTSfMQcGKdMIcR4dv59RoNz1W0=; b=FUk1GzpOusDerY9Z4DWxTBeBAbGxj3JER7CCcyVVwh6XuoaKTbAJIu9glK2RyCaOpW6vzU wGJ3LdQS/zpeIjvWaz5zRtFpM9hMo+44UXlwnL/+HdgwnzeeYNcP0+lHDRt9UUZF+1+nZF MtLgonlwk1slvyVV+kjmhnaBIDe/uM8= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-611-rLkG1hz2Ny20iTo-9QK4BQ-1; Mon, 30 Sep 2024 16:51:15 -0400 X-MC-Unique: rLkG1hz2Ny20iTo-9QK4BQ-1 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-37cd1fb9497so2246466f8f.1 for ; Mon, 30 Sep 2024 13:51:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727729473; x=1728334273; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=osKeptO4NFC9rVaQiTTSfMQcGKdMIcR4dv59RoNz1W0=; b=WiECa7io4sFJ4iJ09vGvIm9q2DpucpznuOyp9hqjKQ4ghyevx9Rg8KilUYy/nqTFts VGSiNSffdkQeyRPZuQN33hOAHheBKVFdfjUjSWAbQxjispiZfD5+ZdcqrW8Y21SCpRIQ C5dbc8fLX7wgc03PayZGNi7C5+CLcFpV2jOwrwNf1rEciSA0wVVAAmNbDhnJCtBha73U ldKrN5shcMhwNAUcTvarVnww3+ks+ULJpttuFFb8nY6d8aYD0XXCC4vlukQm0eibcsfY crLaEbAZg+NLgPYAPaDl6Z2yF3CLMT5IlyPdGT0OViZWQdsLcCUKb9hhUO/82QRECFSV C4Ug== X-Gm-Message-State: AOJu0YwIaJ0qax45Bo1ZLdCXyHbgVlbR97rbnYe1wLLZ9K78K7k3ZVuM 7xc8eaZIF6o+jMfNtGg/u/iruQUEIM7cL2tBrmqdk7iWsyOsY4LHldMdXeaFSpEuwYcHK4GBvGw 1/4sKLbQ9cuys7cktHqBhLKz01YOvFgu8JqamP33vD89/50jA7/6ysTsyFD6TAjF5jKk7sMAxle /zkz8TbCYDZV+wgx0Mwn8Iknsl1B54a4b4gN4xcSo= X-Received: by 2002:a5d:5d82:0:b0:374:bb1a:eebb with SMTP id ffacd0b85a97d-37cf28ada47mr451513f8f.25.1727729473621; Mon, 30 Sep 2024 13:51:13 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGd9tk/5bfU3P/Zgzxsu6KSftuHSWiXxW0gDsbXbotj3cJU/pdH+e6VCz+Uaoj3N/ym87unIQ== X-Received: by 2002:a5d:5d82:0:b0:374:bb1a:eebb with SMTP id ffacd0b85a97d-37cf28ada47mr451504f8f.25.1727729473246; Mon, 30 Sep 2024 13:51:13 -0700 (PDT) Received: from localhost (net-188-216-80-179.cust.vodafonedsl.it. [188.216.80.179]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-37cd565e612sm10226431f8f.42.2024.09.30.13.51.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2024 13:51:12 -0700 (PDT) From: Paolo Valerio To: ovs-dev@openvswitch.org Date: Mon, 30 Sep 2024 22:50:30 +0200 Message-ID: <20240930205034.65484-2-pvalerio@redhat.com> X-Mailer: git-send-email 2.46.1 In-Reply-To: <20240930205034.65484-1-pvalerio@redhat.com> References: <20240930205034.65484-1-pvalerio@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v2 2/6] conntrack: Add zone limit coverage counter. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Similarly to what it's done for conntrack_full, add conntrack_zone_full increased when new entries are not added due to reaching the zone limit. Signed-off-by: Paolo Valerio Acked-by: Aaron Conole --- lib/conntrack.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/conntrack.c b/lib/conntrack.c index e96779e68..8cf200e06 100644 --- a/lib/conntrack.c +++ b/lib/conntrack.c @@ -50,6 +50,7 @@ COVERAGE_DEFINE(conntrack_full); COVERAGE_DEFINE(conntrack_l3csum_err); COVERAGE_DEFINE(conntrack_l4csum_err); COVERAGE_DEFINE(conntrack_lookup_natted_miss); +COVERAGE_DEFINE(conntrack_zone_full); struct conn_lookup_ctx { struct conn_key key; @@ -918,6 +919,7 @@ conn_not_found(struct conntrack *ct, struct dp_packet *pkt, struct zone_limit *zl = zone_limit_lookup_or_default(ct, ctx->key.zone); if (zl && atomic_count_get(&zl->czl.count) >= zl->czl.limit) { + COVERAGE_INC(conntrack_zone_full); return nc; } From patchwork Mon Sep 30 20:50:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valerio X-Patchwork-Id: 1991195 X-Patchwork-Delegate: aconole@redhat.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Bl6Ry/C2; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XHYC2109Zz1xt8 for ; Tue, 1 Oct 2024 06:51:25 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id E0074810F7; Mon, 30 Sep 2024 20:51:23 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id XT50LKer7I1k; Mon, 30 Sep 2024 20:51:22 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 82803810EF Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Bl6Ry/C2 Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 82803810EF; Mon, 30 Sep 2024 20:51:22 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 51F02C0894; Mon, 30 Sep 2024 20:51:22 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id D6203C002A for ; Mon, 30 Sep 2024 20:51:20 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id C5361402C1 for ; Mon, 30 Sep 2024 20:51:20 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id FR7zVTPtg8CD for ; Mon, 30 Sep 2024 20:51:20 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.129.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=pvalerio@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp4.osuosl.org E4233402BB Authentication-Results: smtp4.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org E4233402BB Authentication-Results: smtp4.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=Bl6Ry/C2 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id E4233402BB for ; Mon, 30 Sep 2024 20:51:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1727729478; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8g15JO8SIoLvRY5rRZzaTwBVwUlCNT3fR+CA+tQ0BZI=; b=Bl6Ry/C2JY6T5AuH065HqPiCHpkoAKssC1vY/t+X6JXeXOuq4VBz0nO2Mddq9tV90vNOyQ 60vA3peJnlzP/rCxd2bFI+vhf6ES3zwD23yzmchKTpt57i1yDesjp83l1rp6cGUBm4j5I6 i6TZ2nwqIjWuunGzTDZnpGA+Sh4zP0o= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-617-5A_ybPaaNV20qvPGYtbmtw-1; Mon, 30 Sep 2024 16:51:17 -0400 X-MC-Unique: 5A_ybPaaNV20qvPGYtbmtw-1 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-42cb115566eso48598525e9.2 for ; Mon, 30 Sep 2024 13:51:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727729475; x=1728334275; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=szwWbskAQY3MnlC6LMQivey0e86+4Sz0aq28aSVtIeM=; b=cIMo3vPFWVCFcG9P6FtLpJNztDIgd00GySwgNEItEUZAoOOg3m3Ptu/SevoLSK4lDu OziNcSllZCzMU8bSUeV4ORWxPTXqluG3oXkGtHu6R//G3KvvdjutK39GvO0OyJF9OCwe dHEAXz/5JAkMHygPzRrNOP/JDu1+0ezd/zs8gDRE/BcWQQSl6uhDWR2/9piKjxhNkIpH BEikTU9KxcOdbysryFzVbX9MN6+YWob8ZKicOB2Ei+4lchnVxZLbnuppTWKT0ToCUqy/ XDTkoTRpehLcyB1VSXBtXWAvHYP4LV+ppqoOr0lhoFM9CuvVLdEReOXGQKLUly43LzU7 DVJA== X-Gm-Message-State: AOJu0YzBwBIBQUXOm7dy24i2qE5q/+Ass8PeCrqF3Y/E1rFHDfvnAWrw 7HZ6zEbr47GZ2cml7DaYQUlKVzldIzijP41pwvxt7Ru/PwbnQrHmhRWIbs1wKz6eopSGzodDxcH bIjCDxtEZj0s6etZJycX4a5pAbFU91HYJ5fcjw+xMAh4+/FOyU88lZAUNX+PEI9fTPs4Cjh9/F6 xXfbAgtgCg4S/TEZpY04Ev2NK540gKYTUeJaOUjq4= X-Received: by 2002:a05:600c:1d1e:b0:42b:ac3d:3abc with SMTP id 5b1f17b1804b1-42f584976b0mr134901995e9.24.1727729474808; Mon, 30 Sep 2024 13:51:14 -0700 (PDT) X-Google-Smtp-Source: AGHT+IG4M/SBxgYrQYEMhBamUuP+NaCVZV/sOtfYM25T1jkuneSo6jnB46hmYSp2t4jXbQcPsxlsog== X-Received: by 2002:a05:600c:1d1e:b0:42b:ac3d:3abc with SMTP id 5b1f17b1804b1-42f584976b0mr134901795e9.24.1727729474202; Mon, 30 Sep 2024 13:51:14 -0700 (PDT) Received: from localhost (net-188-216-80-179.cust.vodafonedsl.it. [188.216.80.179]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42e969ffcd5sm162574845e9.26.2024.09.30.13.51.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2024 13:51:13 -0700 (PDT) From: Paolo Valerio To: ovs-dev@openvswitch.org Date: Mon, 30 Sep 2024 22:50:31 +0200 Message-ID: <20240930205034.65484-3-pvalerio@redhat.com> X-Mailer: git-send-email 2.46.1 In-Reply-To: <20240930205034.65484-1-pvalerio@redhat.com> References: <20240930205034.65484-1-pvalerio@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v2 3/6] conntrack: Do not use atomics to report zones info. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Atomics are not needed when reporting zone limits. Remove the restriction by defining a non-atomic common structure to report such data. The change also access atomics using the related operations to retrieve atomics reporting only the fields required by the requesting level instead of relying of struct copy. Signed-off-by: Paolo Valerio --- lib/conntrack-private.h | 8 ++++++++ lib/conntrack.c | 11 ++++++----- lib/conntrack.h | 9 ++++----- lib/dpif-netdev.c | 6 +++--- 4 files changed, 21 insertions(+), 13 deletions(-) diff --git a/lib/conntrack-private.h b/lib/conntrack-private.h index 6c65caa07..2c625d710 100644 --- a/lib/conntrack-private.h +++ b/lib/conntrack-private.h @@ -198,6 +198,14 @@ enum ct_ephemeral_range { #define FOR_EACH_PORT_IN_RANGE(curr, min, max) \ FOR_EACH_PORT_IN_RANGE__(curr, min, max, OVS_JOIN(idx, __COUNTER__)) + +struct conntrack_zone_limit { + int32_t zone; + uint32_t limit; + atomic_count count; + uint32_t zone_limit_seq; /* Used to disambiguate zone limit counts. */ +}; + struct conntrack { struct ovs_mutex ct_lock; /* Protects the following fields. */ struct cmap conns[UINT16_MAX + 1] OVS_GUARDED; diff --git a/lib/conntrack.c b/lib/conntrack.c index 8cf200e06..112d43216 100644 --- a/lib/conntrack.c +++ b/lib/conntrack.c @@ -330,18 +330,19 @@ zone_limit_lookup_or_default(struct conntrack *ct, int32_t zone) return zl ? zl : zone_limit_lookup(ct, DEFAULT_ZONE); } -struct conntrack_zone_limit +struct conntrack_zone_info zone_limit_get(struct conntrack *ct, int32_t zone) { - struct conntrack_zone_limit czl = { + struct conntrack_zone_info czl = { .zone = DEFAULT_ZONE, .limit = 0, - .count = ATOMIC_COUNT_INIT(0), - .zone_limit_seq = 0, + .count = 0, }; struct zone_limit *zl = zone_limit_lookup_or_default(ct, zone); if (zl) { - czl = zl->czl; + czl.zone = zl->czl.zone; + czl.limit = zl->czl.limit; + czl.count = atomic_count_get(&zl->czl.count); } return czl; } diff --git a/lib/conntrack.h b/lib/conntrack.h index 13bb02ea9..c3136e955 100644 --- a/lib/conntrack.h +++ b/lib/conntrack.h @@ -115,11 +115,10 @@ struct conntrack_dump { uint16_t current_zone; }; -struct conntrack_zone_limit { +struct conntrack_zone_info { int32_t zone; uint32_t limit; - atomic_count count; - uint32_t zone_limit_seq; /* Used to disambiguate zone limit counts. */ + unsigned int count; }; struct timeout_policy { @@ -161,8 +160,8 @@ int conntrack_set_sweep_interval(struct conntrack *ct, uint32_t ms); uint32_t conntrack_get_sweep_interval(struct conntrack *ct); bool conntrack_get_tcp_seq_chk(struct conntrack *ct); struct ipf *conntrack_ipf_ctx(struct conntrack *ct); -struct conntrack_zone_limit zone_limit_get(struct conntrack *ct, - int32_t zone); +struct conntrack_zone_info zone_limit_get(struct conntrack *ct, + int32_t zone); int zone_limit_update(struct conntrack *ct, int32_t zone, uint32_t limit); int zone_limit_delete(struct conntrack *ct, int32_t zone); diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c index 3d262463f..2a529f272 100644 --- a/lib/dpif-netdev.c +++ b/lib/dpif-netdev.c @@ -9732,7 +9732,7 @@ dpif_netdev_ct_get_limits(struct dpif *dpif, struct ovs_list *zone_limits_reply) { struct dp_netdev *dp = get_dp_netdev(dpif); - struct conntrack_zone_limit czl; + struct conntrack_zone_info czl; if (!ovs_list_is_empty(zone_limits_request)) { struct ct_dpif_zone_limit *zone_limit; @@ -9741,7 +9741,7 @@ dpif_netdev_ct_get_limits(struct dpif *dpif, if (czl.zone == zone_limit->zone || czl.zone == DEFAULT_ZONE) { ct_dpif_push_zone_limit(zone_limits_reply, zone_limit->zone, czl.limit, - atomic_count_get(&czl.count)); + czl.count); } else { return EINVAL; } @@ -9757,7 +9757,7 @@ dpif_netdev_ct_get_limits(struct dpif *dpif, czl = zone_limit_get(dp->conntrack, z); if (czl.zone == z) { ct_dpif_push_zone_limit(zone_limits_reply, z, czl.limit, - atomic_count_get(&czl.count)); + czl.count); } } } From patchwork Mon Sep 30 20:50:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valerio X-Patchwork-Id: 1991196 X-Patchwork-Delegate: aconole@redhat.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=iRUVCeso; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XHYC91xgVz1xsv for ; Tue, 1 Oct 2024 06:51:33 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id EF43880B7E; Mon, 30 Sep 2024 20:51:29 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id J3LMuO8MJmFB; Mon, 30 Sep 2024 20:51:29 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 066A480C07 Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=iRUVCeso Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 066A480C07; Mon, 30 Sep 2024 20:51:29 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D20FAC002B; Mon, 30 Sep 2024 20:51:28 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1FFF1C0893 for ; Mon, 30 Sep 2024 20:51:27 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 210D560767 for ; Mon, 30 Sep 2024 20:51:24 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 2hud62aQuRAn for ; Mon, 30 Sep 2024 20:51:23 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.129.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=pvalerio@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp3.osuosl.org 11C6260753 Authentication-Results: smtp3.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 11C6260753 Authentication-Results: smtp3.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=iRUVCeso Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp3.osuosl.org (Postfix) with ESMTPS id 11C6260753 for ; Mon, 30 Sep 2024 20:51:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1727729481; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ND7SDJGE1K9bfLzZHo/eIeOOEmq6SPGxq2kYIRcXqo8=; b=iRUVCesouW7mGjIFzJuFHyy04qt2nRHoywWgZENAExaP7zHsqQEmBjj6OmjMFtnpmzlPA5 kgnYi+GDeIRY21FnfCy+GrUypDdvYpOcRjhtnr8o37e6JTqmgBXjDHnoLIIHCAegFYQQLJ MyyeVIhYMTtVlbMJ5dBI6ntG0Fy9h30= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-125-r26KkOsxMiCWwR7papgiwg-1; Mon, 30 Sep 2024 16:51:20 -0400 X-MC-Unique: r26KkOsxMiCWwR7papgiwg-1 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-37ce14d621aso1203235f8f.3 for ; Mon, 30 Sep 2024 13:51:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727729478; x=1728334278; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ND7SDJGE1K9bfLzZHo/eIeOOEmq6SPGxq2kYIRcXqo8=; b=ltTBckTqiuhrPBiRZABLQSYswSmLssvMide0xQEXZR8d57jRmn9aYyySKM8pr7O/DE fBVifmoxOBh2jhYbspPJNQ0ZVKa4VEs4I/w5m6eCu0nbmxfbyeLnn0mYbpenCQfmRcYU 9tb7vs9NfX7noGWCeSZDxkH+zWSecGyXIAQ9x7KKqxNloLWIAcw4ZOS1jTTGHbhT6FR8 AbQnuHwcdvz+q4D1gHAHnkmAsESJYwyd6tJaZiT9ionOZlStx05wJ8BoFBTvcSzh0+pF SBLJwU2z8rRLJ94nx9Oz2IXn/PDoSdN0fkXVlQYzqrsSTrv+EYdnsunZQLO/am8uGOaR aoqQ== X-Gm-Message-State: AOJu0YycGiSK2UbU/1t1thQtNw/0SO4EfKSxsQOwAsmC/x/1U746+7t2 pYK1gDbXmjbBrgtc8e9fLQP5XARyLD+y2aN+ozA+x7BoXV+0C+f1bO//LNa4FM6r31Esu+QCE+L jdsTe7gBej00eUHfEZw29sgaDurzaiShMhQebkWlqQ147m4BWuPzPNTs9jeDFdsMNk21OTKbgBm L9DZqfbbezokn/QjbZ+FzB3Z2MrF/V9+U3sufnco0= X-Received: by 2002:a5d:570e:0:b0:37c:ccbc:6573 with SMTP id ffacd0b85a97d-37cd5ab7301mr6384706f8f.18.1727729477933; Mon, 30 Sep 2024 13:51:17 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGT3cjQqeCE9347TSC4OrRiRo0keE5pCwxfUWi4bSQONT5jbLkSB/zUqQ0wuugcXLfMunw15A== X-Received: by 2002:a5d:570e:0:b0:37c:ccbc:6573 with SMTP id ffacd0b85a97d-37cd5ab7301mr6384687f8f.18.1727729477233; Mon, 30 Sep 2024 13:51:17 -0700 (PDT) Received: from localhost (net-188-216-80-179.cust.vodafonedsl.it. [188.216.80.179]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-37cd564bf8bsm10233049f8f.20.2024.09.30.13.51.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2024 13:51:14 -0700 (PDT) From: Paolo Valerio To: ovs-dev@openvswitch.org Date: Mon, 30 Sep 2024 22:50:32 +0200 Message-ID: <20240930205034.65484-4-pvalerio@redhat.com> X-Mailer: git-send-email 2.46.1 In-Reply-To: <20240930205034.65484-1-pvalerio@redhat.com> References: <20240930205034.65484-1-pvalerio@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v2 4/6] conntrack: Turn zl local limit into atomic. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" while at it, changes struct zone_limit initialization in zone_limit_create() in order to use atomic init operations instead of relying on memset() which, although correctly initializes the struct, is semantically not aware of atomics. Signed-off-by: Paolo Valerio --- v2: - Fixed typo s/semantially/semantically (Aaron) --- lib/conntrack-private.h | 2 +- lib/conntrack.c | 19 ++++++++++++------- 2 files changed, 13 insertions(+), 8 deletions(-) diff --git a/lib/conntrack-private.h b/lib/conntrack-private.h index 2c625d710..2770470d1 100644 --- a/lib/conntrack-private.h +++ b/lib/conntrack-private.h @@ -201,7 +201,7 @@ enum ct_ephemeral_range { struct conntrack_zone_limit { int32_t zone; - uint32_t limit; + atomic_uint32_t limit; atomic_count count; uint32_t zone_limit_seq; /* Used to disambiguate zone limit counts. */ }; diff --git a/lib/conntrack.c b/lib/conntrack.c index 112d43216..3d19d37df 100644 --- a/lib/conntrack.c +++ b/lib/conntrack.c @@ -341,7 +341,7 @@ zone_limit_get(struct conntrack *ct, int32_t zone) struct zone_limit *zl = zone_limit_lookup_or_default(ct, zone); if (zl) { czl.zone = zl->czl.zone; - czl.limit = zl->czl.limit; + atomic_read_relaxed(&zl->czl.limit, &czl.limit); czl.count = atomic_count_get(&zl->czl.count); } return czl; @@ -358,8 +358,9 @@ zone_limit_create(struct conntrack *ct, int32_t zone, uint32_t limit) } if (zone >= DEFAULT_ZONE && zone <= MAX_ZONE) { - zl = xzalloc(sizeof *zl); - zl->czl.limit = limit; + zl = xmalloc(sizeof *zl); + atomic_init(&zl->czl.limit, limit); + atomic_count_init(&zl->czl.count, 0); zl->czl.zone = zone; zl->czl.zone_limit_seq = ct->zone_limit_seq++; uint32_t hash = zone_key_hash(zone, ct->hash_basis); @@ -376,7 +377,7 @@ zone_limit_update(struct conntrack *ct, int32_t zone, uint32_t limit) int err = 0; struct zone_limit *zl = zone_limit_lookup(ct, zone); if (zl) { - zl->czl.limit = limit; + atomic_store_relaxed(&zl->czl.limit, limit); VLOG_INFO("Changed zone limit of %u for zone %d", limit, zone); } else { ovs_mutex_lock(&ct->ct_lock); @@ -916,12 +917,16 @@ conn_not_found(struct conntrack *ct, struct dp_packet *pkt, } if (commit) { + uint32_t czl_limit; struct conn_key_node *fwd_key_node, *rev_key_node; struct zone_limit *zl = zone_limit_lookup_or_default(ct, ctx->key.zone); - if (zl && atomic_count_get(&zl->czl.count) >= zl->czl.limit) { - COVERAGE_INC(conntrack_zone_full); - return nc; + if (zl) { + atomic_read_relaxed(&zl->czl.limit, &czl_limit); + if (atomic_count_get(&zl->czl.count) >= czl_limit) { + COVERAGE_INC(conntrack_zone_full); + return nc; + } } unsigned int n_conn_limit; From patchwork Mon Sep 30 20:50:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valerio X-Patchwork-Id: 1991198 X-Patchwork-Delegate: aconole@redhat.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=eFjR7Cen; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XHYCJ4m4wz1xsv for ; Tue, 1 Oct 2024 06:51:40 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 3B309607C6; Mon, 30 Sep 2024 20:51:38 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id zHkiDOUrskuc; Mon, 30 Sep 2024 20:51:34 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 048A360783 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=eFjR7Cen Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 048A360783; Mon, 30 Sep 2024 20:51:33 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 80682C002B; Mon, 30 Sep 2024 20:51:32 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 1791AC0895 for ; Mon, 30 Sep 2024 20:51:30 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 2C230408C3 for ; Mon, 30 Sep 2024 20:51:27 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 52wS0xRBr6hN for ; Mon, 30 Sep 2024 20:51:24 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.133.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=pvalerio@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp2.osuosl.org 53F83408E8 Authentication-Results: smtp2.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 53F83408E8 Authentication-Results: smtp2.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=eFjR7Cen Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id 53F83408E8 for ; Mon, 30 Sep 2024 20:51:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1727729482; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iRXGHMc2WMQKlF8PUl8tdlcrqeCfMOr+NyVgZ2Zq7dM=; b=eFjR7CenHAVaFnuESqJBJrzE1vtSSfwM2NDShx7MsqbYYGRg4+j18yVt6l3TfmDzNNWVXU tq1jN0WjEaFcLfZTC91IiNbPgtxRjV+0JZwYngdLQ6NYD1Ofg88X9m6IvmPgQydYPURfiV grKNLVTTWqal5ppeXY1PtjY1HScDUMU= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-284-57HmPAauMdatUdIuvqsS4Q-1; Mon, 30 Sep 2024 16:51:21 -0400 X-MC-Unique: 57HmPAauMdatUdIuvqsS4Q-1 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-42ca8037d9aso31617105e9.3 for ; Mon, 30 Sep 2024 13:51:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727729479; x=1728334279; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iRXGHMc2WMQKlF8PUl8tdlcrqeCfMOr+NyVgZ2Zq7dM=; b=spkTquxj/9QraCMt+5b1zWQ0UxGZUyPoy7XQvyX0lPoAf3OtM5fJ/C9nUvK2frZ4Z+ P0DcoTKRXNDUZmCA6jBWqRd+ca0ZAGJPkNtL80IWEPof2RNAfYFtloT0JkwuiLSMihnm ZHv6aGfiTCRA/njCatEYqd3SeyOsWQILr8OYS43RYSpXYGtiXskUBaZy3ASP/1diHlAN ivL3uub6FkRuq4KntMoClsqjbR3UZvc/B/3iUGU9sHo1UJml02WK33xopx8OnDFDMTZY o81LUE5ry8RcXZ+Fr0NoVeSB8VokuX4S/17nNf9ggxwdVpF8u9pBGvllyIUqekJcb+B5 aMmw== X-Gm-Message-State: AOJu0YyyeyKnifqX3Lg3tEVaqmbkBi4dJ6ANAjYDliwsEEWXFpGdM8Ex xB2F4L3s4m/ji4Qua1ZuGcOd4YtVGd5sOqDtq451U72SU/dzgo6cGQCdbdCYf7ODwa5WBQ08DzW /t3oyBSJEK0oqiNm/WumlE/1pYyQTbAxT9kdHhD60XkeDfLsGpSGD5CMHvvCD4wA0zHH56DT0/2 xPiPaZ8zJvMxEVKkeoyIyIHF5vwj4k7avQ4wZpadE= X-Received: by 2002:a05:600c:1d20:b0:42c:b6e4:e3ac with SMTP id 5b1f17b1804b1-42f5840ceedmr86063495e9.3.1727729479109; Mon, 30 Sep 2024 13:51:19 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHoHAbc3EprUmxYhx2RsUCQUPI7NgVPRKo8mkTwzlIig/t+I3yRL8oOi+HR5xI15yO+Afz8Bw== X-Received: by 2002:a05:600c:1d20:b0:42c:b6e4:e3ac with SMTP id 5b1f17b1804b1-42f5840ceedmr86063355e9.3.1727729478489; Mon, 30 Sep 2024 13:51:18 -0700 (PDT) Received: from localhost (net-188-216-80-179.cust.vodafonedsl.it. [188.216.80.179]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-37cd565e881sm10170504f8f.44.2024.09.30.13.51.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2024 13:51:17 -0700 (PDT) From: Paolo Valerio To: ovs-dev@openvswitch.org Date: Mon, 30 Sep 2024 22:50:33 +0200 Message-ID: <20240930205034.65484-5-pvalerio@redhat.com> X-Mailer: git-send-email 2.46.1 In-Reply-To: <20240930205034.65484-1-pvalerio@redhat.com> References: <20240930205034.65484-1-pvalerio@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v2 5/6] conntrack: Use a per zone default limit. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ilya Maximets Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Before this change the default limit, instead of being considered per-zone, was considered as a global value that every new entry was checked against during the creation. This was not the intended behavior as the default limit should be inherited by each zone instead of being an aggregate number. This change corrects that by removing the default limit from the cmap and making it global (atomic). Now, whenever a new connection needs to be committed, if default_zone_limit is set and the entry for the zone doesn't exist, a new entry for that zone is lazily created, marked as default. All subsequent packets for that zone will undergo the regular lookup process. To distinguish between default and user-defined entries, the storage for the limit member of struct conntrack_zone_limit has been changed from a 32-bit unsigned integer to a 64-bit signed integer. The negative value ZONE_LIMIT_CONN_DEFAULT now indicates a default entry. Operations such as creation/deletion are modified accordingly taking into account this new behavior. Worth noting that OVS_REQUIRES(ct->ct_lock) is not a strict requirement for zone_limit_lookup_or_default(), however since the function operates under the lock and it can create an entry in the slow path, the lock requirement is enforced in order to make thread safety checks work. The function can still be moved outside the creation lock or any lock, keeping the fastpath lockless (turning zone_limit_lookup_protected() to its unprotected version) and locking only in the slow path (replacing zone_limit_create__() with zone_limit_create__(). The patch also extends `conntrack - limit by zone` test in order to check the behavior, and while at it, update test's packet-out to use compose-packet function. Fixes: a7f33fdbfb67 ("conntrack: Support zone limits.") Reported-at: https://issues.redhat.com/browse/FDP-122 Reported-by: Ilya Maximets Signed-off-by: Paolo Valerio --- v2: Aaron: - Added entry in NEWS - updated commit message mentioning the storage change for limit --- NEWS | 5 + lib/conntrack-private.h | 7 +- lib/conntrack.c | 233 +++++++++++++++++++++++++++++++--------- tests/system-traffic.at | 64 +++++++---- 4 files changed, 236 insertions(+), 73 deletions(-) diff --git a/NEWS b/NEWS index 7a9626bf4..48384ab1d 100644 --- a/NEWS +++ b/NEWS @@ -1,5 +1,10 @@ Post-v3.4.0 -------------------- + - Userspace datapath: + * The default zone limit, if set, is now inherited by any zone + that does not have a specific value defined, rather than being + treated as a global value, aligning the behavior with that of + the kernel datapath. v3.4.0 - 15 Aug 2024 diff --git a/lib/conntrack-private.h b/lib/conntrack-private.h index 2770470d1..46b212754 100644 --- a/lib/conntrack-private.h +++ b/lib/conntrack-private.h @@ -198,10 +198,11 @@ enum ct_ephemeral_range { #define FOR_EACH_PORT_IN_RANGE(curr, min, max) \ FOR_EACH_PORT_IN_RANGE__(curr, min, max, OVS_JOIN(idx, __COUNTER__)) +#define ZONE_LIMIT_CONN_DEFAULT -1 struct conntrack_zone_limit { int32_t zone; - atomic_uint32_t limit; + atomic_int64_t limit; atomic_count count; uint32_t zone_limit_seq; /* Used to disambiguate zone limit counts. */ }; @@ -212,6 +213,9 @@ struct conntrack { struct rculist exp_lists[N_EXP_LISTS] OVS_GUARDED; struct cmap zone_limits OVS_GUARDED; struct cmap timeout_policies OVS_GUARDED; + uint32_t zone_limit_seq OVS_GUARDED; /* Used to disambiguate zone limit + * counts. */ + atomic_uint32_t default_zone_limit; uint32_t hash_basis; /* Salt for hashing a connection key. */ pthread_t clean_thread; /* Periodically cleans up connection tracker. */ @@ -234,7 +238,6 @@ struct conntrack { * control context. */ struct ipf *ipf; /* Fragmentation handling context. */ - uint32_t zone_limit_seq; /* Used to disambiguate zone limit counts. */ atomic_bool tcp_seq_chk; /* Check TCP sequence numbers. */ atomic_uint32_t sweep_ms; /* Next sweep interval. */ }; diff --git a/lib/conntrack.c b/lib/conntrack.c index 3d19d37df..0061a5636 100644 --- a/lib/conntrack.c +++ b/lib/conntrack.c @@ -270,6 +270,7 @@ conntrack_init(void) atomic_init(&ct->n_conn_limit, DEFAULT_N_CONN_LIMIT); atomic_init(&ct->tcp_seq_chk, true); atomic_init(&ct->sweep_ms, 20000); + atomic_init(&ct->default_zone_limit, 0); latch_init(&ct->clean_thread_exit); ct->clean_thread = ovs_thread_create("ct_clean", clean_thread_main, ct); ct->ipf = ipf_init(); @@ -296,6 +297,28 @@ zone_key_hash(int32_t zone, uint32_t basis) return hash; } +static int64_t +zone_limit_get_limit__(struct conntrack_zone_limit *czl) +{ + int64_t limit; + atomic_read_relaxed(&czl->limit, &limit); + + return limit; +} + +static int64_t +zone_limit_get_limit(struct conntrack *ct, struct conntrack_zone_limit *czl) +{ + int64_t limit = zone_limit_get_limit__(czl); + + if (limit == ZONE_LIMIT_CONN_DEFAULT) { + atomic_read_relaxed(&ct->default_zone_limit, &limit); + limit = limit ? : -1; + } + + return limit; +} + static struct zone_limit * zone_limit_lookup_protected(struct conntrack *ct, int32_t zone) OVS_REQUIRES(ct->ct_lock) @@ -323,11 +346,56 @@ zone_limit_lookup(struct conntrack *ct, int32_t zone) return NULL; } +static struct zone_limit * +zone_limit_create__(struct conntrack *ct, int32_t zone, int64_t limit) + OVS_REQUIRES(ct->ct_lock) +{ + struct zone_limit *zl = NULL; + + if (zone > DEFAULT_ZONE && zone <= MAX_ZONE) { + zl = xmalloc(sizeof *zl); + atomic_init(&zl->czl.limit, limit); + atomic_count_init(&zl->czl.count, 0); + zl->czl.zone = zone; + zl->czl.zone_limit_seq = ct->zone_limit_seq++; + uint32_t hash = zone_key_hash(zone, ct->hash_basis); + cmap_insert(&ct->zone_limits, &zl->node, hash); + } + + return zl; +} + +static struct zone_limit * +zone_limit_create(struct conntrack *ct, int32_t zone, int64_t limit) + OVS_REQUIRES(ct->ct_lock) +{ + struct zone_limit *zl = zone_limit_lookup_protected(ct, zone); + + if (zl) { + return zl; + } + + return zone_limit_create__(ct, zone, limit); +} + +/* Lazily creates a new entry in the zone_limits cmap if default limit + * is set and there's no entry for the zone. */ static struct zone_limit * zone_limit_lookup_or_default(struct conntrack *ct, int32_t zone) + OVS_REQUIRES(ct->ct_lock) { - struct zone_limit *zl = zone_limit_lookup(ct, zone); - return zl ? zl : zone_limit_lookup(ct, DEFAULT_ZONE); + struct zone_limit *zl = zone_limit_lookup_protected(ct, zone); + + if (!zl) { + uint32_t limit; + atomic_read_relaxed(&ct->default_zone_limit, &limit); + + if (limit) { + zl = zone_limit_create__(ct, zone, ZONE_LIMIT_CONN_DEFAULT); + } + } + + return zl; } struct conntrack_zone_info @@ -338,89 +406,147 @@ zone_limit_get(struct conntrack *ct, int32_t zone) .limit = 0, .count = 0, }; - struct zone_limit *zl = zone_limit_lookup_or_default(ct, zone); + struct zone_limit *zl = zone_limit_lookup(ct, zone); if (zl) { - czl.zone = zl->czl.zone; - atomic_read_relaxed(&zl->czl.limit, &czl.limit); + int64_t czl_limit = zone_limit_get_limit__(&zl->czl); + if (czl_limit > ZONE_LIMIT_CONN_DEFAULT) { + czl.zone = zl->czl.zone; + czl.limit = czl_limit; + } else { + atomic_read_relaxed(&ct->default_zone_limit, &czl.limit); + } + czl.count = atomic_count_get(&zl->czl.count); + } else { + atomic_read_relaxed(&ct->default_zone_limit, &czl.limit); } + return czl; } -static int -zone_limit_create(struct conntrack *ct, int32_t zone, uint32_t limit) +static void +zone_limit_clean__(struct conntrack *ct, struct zone_limit *zl) OVS_REQUIRES(ct->ct_lock) { - struct zone_limit *zl = zone_limit_lookup_protected(ct, zone); + uint32_t hash = zone_key_hash(zl->czl.zone, ct->hash_basis); + cmap_remove(&ct->zone_limits, &zl->node, hash); + ovsrcu_postpone(free, zl); +} - if (zl) { - return 0; - } +static void +zone_limit_clean(struct conntrack *ct, struct zone_limit *zl) + OVS_REQUIRES(ct->ct_lock) +{ + uint32_t limit; - if (zone >= DEFAULT_ZONE && zone <= MAX_ZONE) { - zl = xmalloc(sizeof *zl); - atomic_init(&zl->czl.limit, limit); - atomic_count_init(&zl->czl.count, 0); - zl->czl.zone = zone; - zl->czl.zone_limit_seq = ct->zone_limit_seq++; - uint32_t hash = zone_key_hash(zone, ct->hash_basis); - cmap_insert(&ct->zone_limits, &zl->node, hash); - return 0; + atomic_read_relaxed(&ct->default_zone_limit, &limit); + /* Do not remove the entry if the default limit is enabled, but + * simply move the limit to default. */ + if (limit) { + atomic_store_relaxed(&zl->czl.limit, ZONE_LIMIT_CONN_DEFAULT); } else { - return EINVAL; + zone_limit_clean__(ct, zl); } } -int -zone_limit_update(struct conntrack *ct, int32_t zone, uint32_t limit) +static void +zone_limit_clean_default(struct conntrack *ct) + OVS_REQUIRES(ct->ct_lock) { - int err = 0; - struct zone_limit *zl = zone_limit_lookup(ct, zone); - if (zl) { - atomic_store_relaxed(&zl->czl.limit, limit); - VLOG_INFO("Changed zone limit of %u for zone %d", limit, zone); - } else { - ovs_mutex_lock(&ct->ct_lock); - err = zone_limit_create(ct, zone, limit); - ovs_mutex_unlock(&ct->ct_lock); - if (!err) { - VLOG_INFO("Created zone limit of %u for zone %d", limit, zone); - } else { - VLOG_WARN("Request to create zone limit for invalid zone %d", - zone); + struct zone_limit *zl; + int64_t czl_limit; + + atomic_store_relaxed(&ct->default_zone_limit, 0); + + CMAP_FOR_EACH (zl, node, &ct->zone_limits) { + atomic_read_relaxed(&zl->czl.limit, &czl_limit); + if (zone_limit_get_limit__(&zl->czl) == ZONE_LIMIT_CONN_DEFAULT) { + zone_limit_clean__(ct, zl); } } - return err; } -static void -zone_limit_clean(struct conntrack *ct, struct zone_limit *zl) +static bool +zone_limit_delete__(struct conntrack *ct, int32_t zone) OVS_REQUIRES(ct->ct_lock) { - uint32_t hash = zone_key_hash(zl->czl.zone, ct->hash_basis); - cmap_remove(&ct->zone_limits, &zl->node, hash); - ovsrcu_postpone(free, zl); + struct zone_limit *zl = NULL; + + if (zone == DEFAULT_ZONE) { + zone_limit_clean_default(ct); + } else { + zl = zone_limit_lookup_protected(ct, zone); + if (zl) { + zone_limit_clean(ct, zl); + } + } + + return zl != NULL; } int zone_limit_delete(struct conntrack *ct, int32_t zone) { + bool deleted; + ovs_mutex_lock(&ct->ct_lock); - struct zone_limit *zl = zone_limit_lookup_protected(ct, zone); - if (zl) { - zone_limit_clean(ct, zl); - } + deleted = zone_limit_delete__(ct, zone); + ovs_mutex_unlock(&ct->ct_lock); if (zone != DEFAULT_ZONE) { - VLOG_INFO(zl ? "Deleted zone limit for zone %d" - : "Attempted delete of non-existent zone limit: zone %d", + VLOG_INFO(deleted + ? "Deleted zone limit for zone %d" + : "Attempted delete of non-existent zone limit: zone %d", zone); } - ovs_mutex_unlock(&ct->ct_lock); return 0; } +static void +zone_limit_update_default(struct conntrack *ct, int32_t zone, uint32_t limit) +{ + /* limit zero means delete default. */ + if (limit == 0) { + ovs_mutex_lock(&ct->ct_lock); + zone_limit_delete__(ct, zone); + ovs_mutex_unlock(&ct->ct_lock); + } else { + atomic_store_relaxed(&ct->default_zone_limit, limit); + } +} + +int +zone_limit_update(struct conntrack *ct, int32_t zone, uint32_t limit) +{ + struct zone_limit *zl; + int err = 0; + + if (zone == DEFAULT_ZONE) { + zone_limit_update_default(ct, zone, limit); + VLOG_INFO("Set default zone limit to %u", limit); + return err; + } + + zl = zone_limit_lookup(ct, zone); + if (zl) { + atomic_store_relaxed(&zl->czl.limit, limit); + VLOG_INFO("Changed zone limit of %u for zone %d", limit, zone); + } else { + ovs_mutex_lock(&ct->ct_lock); + err = zone_limit_create(ct, zone, limit) == NULL; + ovs_mutex_unlock(&ct->ct_lock); + if (!err) { + VLOG_INFO("Created zone limit of %u for zone %d", limit, zone); + } else { + VLOG_WARN("Request to create zone limit for invalid zone %d", + zone); + } + } + + return err; +} + static void conn_clean__(struct conntrack *ct, struct conn *conn) OVS_REQUIRES(ct->ct_lock) @@ -917,13 +1043,14 @@ conn_not_found(struct conntrack *ct, struct dp_packet *pkt, } if (commit) { - uint32_t czl_limit; + int64_t czl_limit; struct conn_key_node *fwd_key_node, *rev_key_node; struct zone_limit *zl = zone_limit_lookup_or_default(ct, ctx->key.zone); if (zl) { - atomic_read_relaxed(&zl->czl.limit, &czl_limit); - if (atomic_count_get(&zl->czl.count) >= czl_limit) { + czl_limit = zone_limit_get_limit(ct, &zl->czl); + if (czl_limit >= 0 && + atomic_count_get(&zl->czl.count) >= czl_limit) { COVERAGE_INC(conntrack_zone_full); return nc; } diff --git a/tests/system-traffic.at b/tests/system-traffic.at index 158629f42..fe115d92b 100644 --- a/tests/system-traffic.at +++ b/tests/system-traffic.at @@ -5686,6 +5686,41 @@ priority=100,in_port=2,udp,action=ct(zone=3,commit),1 AT_CHECK([ovs-ofctl --bundle add-flows br0 flows.txt]) +m4_define([UDP_PKT], [m4_join([,], + [eth_src=50:54:00:00:00:0$1,eth_dst=50:54:00:00:00:0$2,dl_type=0x0800], + [nw_src=10.1.1.$1,nw_dst=10.1.1.$2], + [nw_proto=17,nw_ttl=64,nw_frag=no], + [udp_src=1,udp_dst=$3])]) + +AT_CHECK([ovs-appctl dpctl/ct-set-limits default=3]) +AT_CHECK([ovs-appctl dpctl/ct-get-limits], [],[dnl +default limit=3 +]) + +dnl Send 5 packets from each port in order to hit the default zone limit. +for i in $(seq 2 6); do + pkt=$(ovs-ofctl compose-packet --bare "UDP_PKT([1], [2], [$i])") + AT_CHECK([ovs-ofctl -O OpenFlow13 packet-out br0 "in_port=1 packet=${pkt} actions=resubmit(,0)"]) +done + +AT_CHECK([ovs-appctl dpctl/ct-get-limits zone=1], [],[dnl +default limit=3 +zone=1,limit=3,count=3 +]) + +for i in $(seq 2 6); do + pkt=$(ovs-ofctl compose-packet --bare "UDP_PKT([3], [4], [$i])") + AT_CHECK([ovs-ofctl -O OpenFlow13 packet-out br0 "in_port=2 packet=${pkt} actions=resubmit(,0)"]) +done + +AT_CHECK([ovs-appctl dpctl/ct-get-limits zone=1,3], [],[dnl +default limit=3 +zone=1,limit=3,count=3 +zone=3,limit=3,count=3 +]) + +AT_CHECK([ovs-appctl dpctl/flush-conntrack]) + AT_CHECK([ovs-appctl dpctl/ct-set-limits default=10 zone=1,limit=5 zone=2,limit=3 zone=3,limit=3 zone=4,limit=15]) AT_CHECK([ovs-appctl dpctl/ct-del-limits zone=2,4,5]) AT_CHECK([ovs-appctl dpctl/ct-get-limits zone=1,2,3,4], [],[dnl @@ -5697,15 +5732,10 @@ zone=4,limit=10,count=0 ]) dnl Test UDP from port 1 -AT_CHECK([ovs-ofctl -O OpenFlow13 packet-out br0 "in_port=1 packet=50540000000a50540000000908004500001c000000000011a4cd0a0101010a0101020001000200080000 actions=resubmit(,0)"]) -AT_CHECK([ovs-ofctl -O OpenFlow13 packet-out br0 "in_port=1 packet=50540000000a50540000000908004500001c000000000011a4cd0a0101010a0101020001000300080000 actions=resubmit(,0)"]) -AT_CHECK([ovs-ofctl -O OpenFlow13 packet-out br0 "in_port=1 packet=50540000000a50540000000908004500001c000000000011a4cd0a0101010a0101020001000400080000 actions=resubmit(,0)"]) -AT_CHECK([ovs-ofctl -O OpenFlow13 packet-out br0 "in_port=1 packet=50540000000a50540000000908004500001c000000000011a4cd0a0101010a0101020001000500080000 actions=resubmit(,0)"]) -AT_CHECK([ovs-ofctl -O OpenFlow13 packet-out br0 "in_port=1 packet=50540000000a50540000000908004500001c000000000011a4cd0a0101010a0101020001000600080000 actions=resubmit(,0)"]) -AT_CHECK([ovs-ofctl -O OpenFlow13 packet-out br0 "in_port=1 packet=50540000000a50540000000908004500001c000000000011a4cd0a0101010a0101020001000700080000 actions=resubmit(,0)"]) -AT_CHECK([ovs-ofctl -O OpenFlow13 packet-out br0 "in_port=1 packet=50540000000a50540000000908004500001c000000000011a4cd0a0101010a0101020001000800080000 actions=resubmit(,0)"]) -AT_CHECK([ovs-ofctl -O OpenFlow13 packet-out br0 "in_port=1 packet=50540000000a50540000000908004500001c000000000011a4cd0a0101010a0101020001000900080000 actions=resubmit(,0)"]) -AT_CHECK([ovs-ofctl -O OpenFlow13 packet-out br0 "in_port=1 packet=50540000000a50540000000908004500001c000000000011a4cd0a0101010a0101020001000a00080000 actions=resubmit(,0)"]) +for i in $(seq 2 10); do + pkt=$(ovs-ofctl compose-packet --bare "UDP_PKT([1], [2], [$i])") + AT_CHECK([ovs-ofctl -O OpenFlow13 packet-out br0 "in_port=1 packet=${pkt} actions=resubmit(,0)"]) +done AT_CHECK([ovs-appctl dpctl/ct-get-limits zone=1,2,3,4,5], [0], [dnl default limit=10 @@ -5732,11 +5762,10 @@ udp,orig=(src=10.1.1.1,dst=10.1.1.2,sport=1,dport=6),reply=(src=10.1.1.2,dst=10. ]) dnl Test UDP from port 2 -AT_CHECK([ovs-ofctl -O OpenFlow13 packet-out br0 "in_port=2 packet=50540000000a50540000000908004500001c000000000011a4c90a0101030a0101040001000200080000 actions=resubmit(,0)"]) -AT_CHECK([ovs-ofctl -O OpenFlow13 packet-out br0 "in_port=2 packet=50540000000a50540000000908004500001c000000000011a4c90a0101030a0101040001000300080000 actions=resubmit(,0)"]) -AT_CHECK([ovs-ofctl -O OpenFlow13 packet-out br0 "in_port=2 packet=50540000000a50540000000908004500001c000000000011a4c90a0101030a0101040001000400080000 actions=resubmit(,0)"]) -AT_CHECK([ovs-ofctl -O OpenFlow13 packet-out br0 "in_port=2 packet=50540000000a50540000000908004500001c000000000011a4c90a0101030a0101040001000500080000 actions=resubmit(,0)"]) -AT_CHECK([ovs-ofctl -O OpenFlow13 packet-out br0 "in_port=2 packet=50540000000a50540000000908004500001c000000000011a4c90a0101030a0101040001000600080000 actions=resubmit(,0)"]) +for i in $(seq 2 6); do + pkt=$(ovs-ofctl compose-packet --bare "UDP_PKT([3], [4], [$i])") + AT_CHECK([ovs-ofctl -O OpenFlow13 packet-out br0 "in_port=2 packet=${pkt} actions=resubmit(,0)"]) +done AT_CHECK([ovs-appctl dpctl/ct-get-limits zone=1,3], [0], [dnl default limit=10 @@ -5797,10 +5826,9 @@ default limit=10 zone=1,limit=3,count=0 zone=3,limit=3,count=0]) -for i in 2 3 4 5 6; do - packet="50540000000a50540000000908004500001c000000000011a4c90a0101030a0101040001000${i}00080000" - AT_CHECK([ovs-ofctl -O OpenFlow13 packet-out br0 \ - "in_port=2 packet=${packet} actions=resubmit(,0)"]) +for i in $(seq 2 6); do + pkt=$(ovs-ofctl compose-packet --bare "UDP_PKT([3], [4], [$i])") + AT_CHECK([ovs-ofctl -O OpenFlow13 packet-out br0 "in_port=2 packet=${pkt} actions=resubmit(,0)"]) done AT_CHECK([ovs-appctl dpctl/dump-conntrack | grep "orig=.src=10\.1\.1\.3," | sort ], [0], [dnl From patchwork Mon Sep 30 20:50:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Valerio X-Patchwork-Id: 1991197 X-Patchwork-Delegate: aconole@redhat.com Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=AkH7sOkp; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::136; helo=smtp3.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4XHYCF4kjkz1xsv for ; Tue, 1 Oct 2024 06:51:37 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 9B9786079F; Mon, 30 Sep 2024 20:51:35 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id 7ngJQu_ptDc9; Mon, 30 Sep 2024 20:51:32 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=140.211.9.56; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 5472760767 Authentication-Results: smtp3.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=AkH7sOkp Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 5472760767; Mon, 30 Sep 2024 20:51:30 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 823B7C0894; Mon, 30 Sep 2024 20:51:30 +0000 (UTC) X-Original-To: ovs-dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 0B188C002B for ; Mon, 30 Sep 2024 20:51:28 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id F0EC5802CC for ; Mon, 30 Sep 2024 20:51:25 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id oA_me5jbSL2j for ; Mon, 30 Sep 2024 20:51:25 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.129.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=pvalerio@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp1.osuosl.org 000C980AB6 Authentication-Results: smtp1.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 000C980AB6 Authentication-Results: smtp1.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=AkH7sOkp Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id 000C980AB6 for ; Mon, 30 Sep 2024 20:51:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1727729483; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=17qnUoQyMYZkX3gCPBj7d3ZIcGpCgwjuI/FZFb0+1ms=; b=AkH7sOkpqr9zIZq3T+Nv6gYRuWGOps0W6VdoDTeLp7zpaXcCFVcMng++C9kthLSTp9QzFa 0a7yahu6N/Z28ydRnt4pzxgiIMsgY6UnJl3kOqClAVZhv6du9z98Jaj+S1UXLIofdNBEWV 65F0kiP+n1u3a2dEQ1kcPO9Iq1djNSE= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-331-1q7xJb0QNGuk2cEXOgIIgA-1; Mon, 30 Sep 2024 16:51:22 -0400 X-MC-Unique: 1q7xJb0QNGuk2cEXOgIIgA-1 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-42e77b5d3dcso31673485e9.1 for ; Mon, 30 Sep 2024 13:51:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727729480; x=1728334280; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=17qnUoQyMYZkX3gCPBj7d3ZIcGpCgwjuI/FZFb0+1ms=; b=k0k90ijqHKfuLM0VBYcRfPqG0kz50au3+RFND2dlxTK2eePw6o8cVsBOJ1JJ93kgGv RocmWn1vL+F5LgSfnufvPU+5gesFw/HDkMehCKy9suwoHGuQxV5mmgk/aHdGoQANQcgS w4o9qdBUS8/a6uheT5HhVCmeZQiu3aDu+4ziQPcC4ifvlI2VJcD8SNsLjgiOixQ6NBU5 iZrngbPNGMpmvGkusBaCeP0GErUhpmottIPj3nT/U/gsFxqQM1rnBjy5zcfja9fheEU3 XmLpXNZQV/0XJUos+dUQp79Y1PNiKkFy3Bzt2JwOCULq89b4k6S9lmiBCOTXy6HhK2PJ qcpQ== X-Gm-Message-State: AOJu0YzAcSq5HoL/GCbPnEd6Gu0nwmqa5F5bHigSY8VLuanBLkebyDOq t632IDyzcwj0ksGDQoS4fcNfpx7iaATEufHJmDr5ov3fcTZsmy/cPMq57GdFxU0uObj3/aGxuOU fWIGalZuULfUVYjrhXRiEHOyBrxsYEgqof+/LJcLTLG5Crhw1NBtW05r1Z0703RqMABYjdywdDx 2aTXzDyAaonJcudTujKQLuGPCIdHnZIZkUQhRxoO4= X-Received: by 2002:a05:600c:4751:b0:42c:bf94:f9a6 with SMTP id 5b1f17b1804b1-42f58485d75mr94529355e9.26.1727729480217; Mon, 30 Sep 2024 13:51:20 -0700 (PDT) X-Google-Smtp-Source: AGHT+IE9thUoHZ9XRW+hxFOmEvvg4d9MU2VNd2+v25ZfuXAy3ZW+TECsCOTvGL0T6eNMJol/FR1CnA== X-Received: by 2002:a05:600c:4751:b0:42c:bf94:f9a6 with SMTP id 5b1f17b1804b1-42f58485d75mr94529235e9.26.1727729479673; Mon, 30 Sep 2024 13:51:19 -0700 (PDT) Received: from localhost (net-188-216-80-179.cust.vodafonedsl.it. [188.216.80.179]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-42e969f23d5sm166123785e9.13.2024.09.30.13.51.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2024 13:51:19 -0700 (PDT) From: Paolo Valerio To: ovs-dev@openvswitch.org Date: Mon, 30 Sep 2024 22:50:34 +0200 Message-ID: <20240930205034.65484-6-pvalerio@redhat.com> X-Mailer: git-send-email 2.46.1 In-Reply-To: <20240930205034.65484-1-pvalerio@redhat.com> References: <20240930205034.65484-1-pvalerio@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v2 6/6] dpctl: Do not allow out of range values in ct-set-limits. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" The ovs_scan() doesn't enforce in-range values and so lsbits are stored in case of out-of-range or negative values. This way negative or values greater than MAX_UINT32 for "default" are all accepted in dpctl_ct_set_limits(), but they will eventually be casted to uint32_t, whereas for zones all the values above are considered invalid. Align their behaviors and extend the tests for checking values out of the range. Signed-off-by: Paolo Valerio --- lib/dpctl.c | 5 +++-- tests/system-traffic.at | 42 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+), 2 deletions(-) diff --git a/lib/dpctl.c b/lib/dpctl.c index 77bf4bf53..2a700f24a 100644 --- a/lib/dpctl.c +++ b/lib/dpctl.c @@ -2169,8 +2169,8 @@ dpctl_ct_set_limits(int argc, const char *argv[], struct ovs_list zone_limits = OVS_LIST_INITIALIZER(&zone_limits); int i = dp_arg_exists(argc, argv) ? 2 : 1; struct ds ds = DS_EMPTY_INITIALIZER; + unsigned long long default_limit; struct dpif *dpif = NULL; - uint32_t default_limit; int error; if (i >= argc) { @@ -2186,7 +2186,8 @@ dpctl_ct_set_limits(int argc, const char *argv[], /* Parse default limit */ if (!strncmp(argv[i], "default=", 8)) { - if (ovs_scan(argv[i], "default=%"SCNu32, &default_limit)) { + if (str_to_ullong(argv[i] + 8, 10, &default_limit) && + default_limit <= UINT32_MAX) { ct_dpif_push_zone_limit(&zone_limits, OVS_ZONE_LIMIT_DEFAULT_ZONE, default_limit, 0); i++; diff --git a/tests/system-traffic.at b/tests/system-traffic.at index fe115d92b..bcb08b0e8 100644 --- a/tests/system-traffic.at +++ b/tests/system-traffic.at @@ -5686,12 +5686,54 @@ priority=100,in_port=2,udp,action=ct(zone=3,commit),1 AT_CHECK([ovs-ofctl --bundle add-flows br0 flows.txt]) +dnl Test values out of range for the default limit. +dnl Try to set a negative value. +AT_CHECK([ovs-appctl dpctl/ct-set-limits default=-1], [2], [ignore], [dnl +ovs-vswitchd: invalid default limit (Invalid argument) +ovs-appctl: ovs-vswitchd: server returned an error +]) + +dnl Try to set UINT32_MAX. +AT_CHECK([ovs-appctl dpctl/ct-set-limits default=4294967296], [2], [ignore], [dnl +ovs-vswitchd: invalid default limit (Invalid argument) +ovs-appctl: ovs-vswitchd: server returned an error +]) + +dnl Same range checks for zones. +AT_CHECK([ovs-appctl dpctl/ct-set-limits zone=1,limit=-1], [2], [ignore], [dnl +ovs-vswitchd: failed to parse field limit (Invalid argument) +ovs-appctl: ovs-vswitchd: server returned an error +]) + +AT_CHECK([ovs-appctl dpctl/ct-set-limits zone=1,limit=4294967296], [2], [ignore], [dnl +ovs-vswitchd: failed to parse field limit (Invalid argument) +ovs-appctl: ovs-vswitchd: server returned an error +]) + +dnl Double check no limits have been applied. +AT_CHECK([ovs-appctl dpctl/ct-get-limits], [],[dnl +default limit=0 +]) + m4_define([UDP_PKT], [m4_join([,], [eth_src=50:54:00:00:00:0$1,eth_dst=50:54:00:00:00:0$2,dl_type=0x0800], [nw_src=10.1.1.$1,nw_dst=10.1.1.$2], [nw_proto=17,nw_ttl=64,nw_frag=no], [udp_src=1,udp_dst=$3])]) +AT_CHECK([ovs-appctl dpctl/ct-set-limits zone=1,limit=0]) +pkt=$(ovs-ofctl compose-packet --bare "UDP_PKT([1], [2], [2])") +AT_CHECK([ovs-ofctl -O OpenFlow13 packet-out br0 "in_port=1 packet=${pkt} actions=resubmit(,0)"]) + +dnl Double check the zl entry exists but no connection was added. +AT_CHECK([ovs-appctl dpctl/ct-get-limits], [],[dnl +default limit=0 +zone=1,limit=0,count=0 +]) + +dnl Remove limit for zone=1. +AT_CHECK([ovs-appctl dpctl/ct-del-limits zone=1]) + AT_CHECK([ovs-appctl dpctl/ct-set-limits default=3]) AT_CHECK([ovs-appctl dpctl/ct-get-limits], [],[dnl default limit=3