From patchwork Thu May 4 17:36:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Traynor X-Patchwork-Id: 1776994 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=openvswitch.org (client-ip=140.211.166.133; helo=smtp2.osuosl.org; envelope-from=ovs-dev-bounces@openvswitch.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=h/7qXWbw; dkim-atps=neutral Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4QC1Fs0wlMz20fg for ; Fri, 5 May 2023 03:36:33 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id C5C51429BC; Thu, 4 May 2023 17:36:30 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org C5C51429BC Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=h/7qXWbw X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IFf6nF2Ok5HT; Thu, 4 May 2023 17:36:29 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 9AB01429CB; Thu, 4 May 2023 17:36:28 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 9AB01429CB Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 7D7EDC0037; Thu, 4 May 2023 17:36:28 +0000 (UTC) X-Original-To: dev@openvswitch.org Delivered-To: ovs-dev@lists.linuxfoundation.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id DDB3DC0037 for ; Thu, 4 May 2023 17:36:27 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id B64024275E for ; Thu, 4 May 2023 17:36:27 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org B64024275E Authentication-Results: smtp4.osuosl.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=h/7qXWbw X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5lZI6RjBqWUv for ; Thu, 4 May 2023 17:36:26 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp4.osuosl.org 159204276D Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id 159204276D for ; Thu, 4 May 2023 17:36:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1683221784; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=f0+PyrFyDOM9hC8Ji2+6zzjmtjnmGWjSeERKJRziNL0=; b=h/7qXWbwL0Wyxx2Vt5Z57DbT9MadWK3fDMzCoL1vLwmrmdJBtDVMABGKQiCfWf41/ktqMi 8yoY7+rOriCsTFl3Hum+m2lb1tJisNUVqwCxQOhh/dADzzCcuWBoLQBHIwS1CRFvBKWBd6 8hpJ5jAJ1dsgBObQlIWseqrDJHO2Khs= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-396-2YeWbLQQNtSS435SIkG9DQ-1; Thu, 04 May 2023 13:36:22 -0400 X-MC-Unique: 2YeWbLQQNtSS435SIkG9DQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C188310504A0; Thu, 4 May 2023 17:36:21 +0000 (UTC) Received: from rh.redhat.com (unknown [10.39.192.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6CF3C4020BF0; Thu, 4 May 2023 17:36:20 +0000 (UTC) From: Kevin Traynor To: dev@openvswitch.org Date: Thu, 4 May 2023 18:36:06 +0100 Message-Id: <20230504173606.262663-3-ktraynor@redhat.com> In-Reply-To: <20230504173606.262663-1-ktraynor@redhat.com> References: <20230504173606.262663-1-ktraynor@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: simon.horman@corigine.com, david.marchand@redhat.com, i.maximets@ovn.org Subject: [ovs-dev] [PATCH v3 2/2] netdev-dpdk: Check rx/tx descriptor sizes for device. X-BeenThere: ovs-dev@openvswitch.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ovs-dev-bounces@openvswitch.org Sender: "dev" By default OVS configures 2048 descriptors for tx and rx queues on DPDK devices. It also allows the user to configure those values. If the values used are not acceptable to the device then queue setup would fail. The device exposes it's max/min/alignment requirements and OVS applies some limits also. Use these to ensure an acceptable value is used for the number of descriptors on a device tx/rx. If the default or user value is not acceptable, adjust to a suitable value and log. Reported-at: https://bugzilla.redhat.com/2119876 Signed-off-by: Kevin Traynor --- lib/netdev-dpdk.c | 53 +++++++++++++++++++++++++++++++++++++---------- 1 file changed, 42 insertions(+), 11 deletions(-) diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index 2d9afc493..0ea6055bc 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -1911,8 +1911,27 @@ dpdk_set_rxq_config(struct netdev_dpdk *dev, const struct smap *args) static void dpdk_process_queue_size(struct netdev *netdev, const struct smap *args, - const char *flag, int default_size, int *new_size) + struct rte_eth_dev_info *info, bool is_rx) { - int queue_size = smap_get_int(args, flag, default_size); + struct netdev_dpdk *dev = netdev_dpdk_cast(netdev); + struct rte_eth_desc_lim *lim; + int default_size, queue_size; + int *requested_size; + if (is_rx) { + default_size = NIC_PORT_DEFAULT_RXQ_SIZE; + queue_size = smap_get_int(args, "n_rxq_desc", default_size); + requested_size = &dev->requested_rxq_size; + lim = info ? &info->rx_desc_lim : NULL; + + } else { + default_size = NIC_PORT_DEFAULT_TXQ_SIZE; + queue_size = smap_get_int(args, "n_txq_desc", default_size); + requested_size = &dev->requested_txq_size; + lim = info ? &info->tx_desc_lim : NULL; + } + + *requested_size = queue_size; + + /* Check for OVS limits. */ if (queue_size <= 0 || queue_size > NIC_PORT_MAX_Q_SIZE || !is_pow2(queue_size)) { @@ -1920,6 +1939,18 @@ dpdk_process_queue_size(struct netdev *netdev, const struct smap *args, } - if (queue_size != *new_size) { - *new_size = queue_size; + if (lim) { + /* Check for device limits. */ + if (lim->nb_align) { + queue_size = ROUND_UP(queue_size, lim->nb_align); + } + queue_size = MIN(queue_size, lim->nb_max); + queue_size = MAX(queue_size, lim->nb_min); + } + + if (*requested_size != queue_size) { + VLOG_INFO("Interface %s cannot set %s descriptor size to %d. " + "Adjusted to %d.", dev->up.name, is_rx ? "rx": "tx" , + *requested_size, queue_size); + *requested_size = queue_size; netdev_request_reconfigure(netdev); } @@ -1938,7 +1969,9 @@ netdev_dpdk_set_config(struct netdev *netdev, const struct smap *args, {RTE_ETH_FC_RX_PAUSE, RTE_ETH_FC_FULL } }; + struct rte_eth_dev_info info; const char *new_devargs; const char *vf_mac; int err = 0; + int ret; ovs_mutex_lock(&dpdk_mutex); @@ -1947,11 +1980,4 @@ netdev_dpdk_set_config(struct netdev *netdev, const struct smap *args, dpdk_set_rxq_config(dev, args); - dpdk_process_queue_size(netdev, args, "n_rxq_desc", - NIC_PORT_DEFAULT_RXQ_SIZE, - &dev->requested_rxq_size); - dpdk_process_queue_size(netdev, args, "n_txq_desc", - NIC_PORT_DEFAULT_TXQ_SIZE, - &dev->requested_txq_size); - new_devargs = smap_get(args, "dpdk-devargs"); @@ -2073,4 +2099,9 @@ netdev_dpdk_set_config(struct netdev *netdev, const struct smap *args, } + ret = rte_eth_dev_info_get(dev->port_id, &info); + + dpdk_process_queue_size(netdev, args, !ret ? &info : NULL, true); + dpdk_process_queue_size(netdev, args, !ret ? &info : NULL, false); + out: ovs_mutex_unlock(&dev->mutex);