From patchwork Fri Jun 14 15:08:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Marchand X-Patchwork-Id: 1947951 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=KmoX5q1c; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=2605:bc80:3010::138; helo=smtp1.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=patchwork.ozlabs.org) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (secp384r1) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4W12jg11sQz20X9 for ; Sat, 15 Jun 2024 01:08:53 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id C42A480AE1; Fri, 14 Jun 2024 15:08:50 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id bagwSpoF0bqw; Fri, 14 Jun 2024 15:08:49 +0000 (UTC) X-Comment: SPF check N/A for local connections - client-ip=2605:bc80:3010:104::8cd3:938; helo=lists.linuxfoundation.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver= DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org B229080AD0 Authentication-Results: smtp1.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key, unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=KmoX5q1c Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTPS id B229080AD0; Fri, 14 Jun 2024 15:08:49 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 7F8BEC0012; Fri, 14 Jun 2024 15:08:49 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 9E40DC0011 for ; Fri, 14 Jun 2024 15:08:48 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 7426E40095 for ; Fri, 14 Jun 2024 15:08:48 +0000 (UTC) X-Virus-Scanned: amavis at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavis, port 10024) with ESMTP id yGx0zuzfKeJp for ; Fri, 14 Jun 2024 15:08:47 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=170.10.129.124; helo=us-smtp-delivery-124.mimecast.com; envelope-from=david.marchand@redhat.com; receiver= DMARC-Filter: OpenDMARC Filter v1.4.2 smtp4.osuosl.org DDDA440091 Authentication-Results: smtp4.osuosl.org; dmarc=pass (p=none dis=none) header.from=redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org DDDA440091 Authentication-Results: smtp4.osuosl.org; dkim=pass (1024-bit key, unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=KmoX5q1c Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id DDDA440091 for ; Fri, 14 Jun 2024 15:08:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1718377725; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=rN6XLKwQqe3jQUyhnlZcrbTXRCvSrZU7+LbjL84RxDE=; b=KmoX5q1cL+r1/sYF8CfEHGNGN2wQZ1junsjS9kwFCO4IuFqhvg2uQtNHuzXOQ4z6QlV5fl Y/9Pir4qInnkcNdp8l8bY276/oaTFZ0qyT6lWVsFEgjdIwEgSKaxCQE2vrF0ZMxUbu/jLQ Que4sPJyKa+mwpOPO6C6eQYVvFBjov0= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-263-JX-aofMyMsa5UOQ-c7I2Fg-1; Fri, 14 Jun 2024 11:08:44 -0400 X-MC-Unique: JX-aofMyMsa5UOQ-c7I2Fg-1 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8BF441956089 for ; Fri, 14 Jun 2024 15:08:43 +0000 (UTC) Received: from dmarchan.redhat.com (unknown [10.45.225.89]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 7520C1956056; Fri, 14 Jun 2024 15:08:41 +0000 (UTC) From: David Marchand To: dev@openvswitch.org Date: Fri, 14 Jun 2024 17:08:37 +0200 Message-ID: <20240614150837.2027561-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Subject: [ovs-dev] [PATCH v3] netdev-dpdk: Use LSC interrupt mode. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: maxime.coquelin@redhat.com Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" Querying link status may get delayed for an undeterministic (long) time with mlx5 ports. This is a consequence of the mlx5 driver calling ethtool kernel API and getting stuck on the kernel RTNL lock while some other operation is in progress under this lock. One impact for long link status query is that it is called under the bond lock taken in write mode periodically in bond_run(). In parallel, datapath threads may block requesting to read bonding related info (like for example in bond_check_admissibility()). The LSC interrupt mode is available with many DPDK drivers and is used by default with testpmd. It seems safe enough to switch on this feature by default in OVS. We keep the per interface option to disable this feature in case of an unforeseen bug. Signed-off-by: David Marchand Reviewed-by: Robin Jarry Acked-by: Mike Pattrick --- Changes since v2: - fixed typo in NEWS, Changes since v1: - (early) fail when interrupt lsc is requested by user but not supported by the driver, - otherwise, log a debug message if user did not request interrupt mode, --- Documentation/topics/dpdk/phy.rst | 4 ++-- NEWS | 3 +++ lib/netdev-dpdk.c | 13 ++++++++++++- vswitchd/vswitch.xml | 8 ++++---- 4 files changed, 21 insertions(+), 7 deletions(-) diff --git a/Documentation/topics/dpdk/phy.rst b/Documentation/topics/dpdk/phy.rst index efd168cba8..eefc25613d 100644 --- a/Documentation/topics/dpdk/phy.rst +++ b/Documentation/topics/dpdk/phy.rst @@ -546,8 +546,8 @@ the firmware every time to fulfil this request. Note that not all PMD drivers support LSC interrupts. -The default configuration is polling mode. To set interrupt mode, option -``dpdk-lsc-interrupt`` has to be set to ``true``. +The default configuration is interrupt mode. To set polling mode, option +``dpdk-lsc-interrupt`` has to be set to ``false``. Command to set interrupt mode for a specific interface:: $ ovs-vsctl set interface options:dpdk-lsc-interrupt=true diff --git a/NEWS b/NEWS index 5ae0108d55..d05f2d0f89 100644 --- a/NEWS +++ b/NEWS @@ -9,6 +9,9 @@ Post-v3.3.0 https://github.com/openvswitch/ovs.git - DPDK: * OVS validated with DPDK 23.11.1. + * Link status changes are now handled via interrupt mode if the DPDK + driver supports it. It is possible to revert to polling mode by setting + per interface 'options:dpdk-lsc-interrupt' to 'false'. v3.3.0 - 16 Feb 2024 diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 0fa37d5145..a260bc8485 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -2397,7 +2397,18 @@ netdev_dpdk_set_config(struct netdev *netdev, const struct smap *args, } } - lsc_interrupt_mode = smap_get_bool(args, "dpdk-lsc-interrupt", false); + lsc_interrupt_mode = smap_get_bool(args, "dpdk-lsc-interrupt", true); + if (lsc_interrupt_mode && !(*info.dev_flags & RTE_ETH_DEV_INTR_LSC)) { + if (smap_get(args, "dpdk-lsc-interrupt")) { + VLOG_ERR("interface '%s': link status interrupt is not supported.", + netdev_get_name(netdev)); + err = EINVAL; + goto out; + } + VLOG_DBG("interface '%s': not enabling link status interrupt.", + netdev_get_name(netdev)); + lsc_interrupt_mode = false; + } if (dev->requested_lsc_interrupt_mode != lsc_interrupt_mode) { dev->requested_lsc_interrupt_mode = lsc_interrupt_mode; netdev_request_reconfigure(netdev); diff --git a/vswitchd/vswitch.xml b/vswitchd/vswitch.xml index 8a1b607d71..e3afb78a4e 100644 --- a/vswitchd/vswitch.xml +++ b/vswitchd/vswitch.xml @@ -4647,12 +4647,12 @@ ovs-vsctl add-port br0 p0 -- set Interface p0 type=patch options:peer=p1 \

- Set this value to true to configure interrupt mode for - Link State Change (LSC) detection instead of poll mode for the DPDK - interface. + Set this value to false to configure poll mode for + Link State Change (LSC) detection instead of interrupt mode for the + DPDK interface.

- If this value is not set, poll mode is configured. + If this value is not set, interrupt mode is configured.

This parameter has an effect only on netdev dpdk interfaces.