From patchwork Tue Oct 11 05:56:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Khalid Elmously X-Patchwork-Id: 1688467 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=o2g53LqH; dkim-atps=neutral Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4MmlRz2lY7z23k2 for ; Tue, 11 Oct 2022 16:57:33 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1oi8GV-0008Gs-F6; Tue, 11 Oct 2022 05:57:19 +0000 Received: from smtp-relay-internal-1.internal ([10.131.114.114] helo=smtp-relay-internal-1.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1oi8GQ-0008FI-Lc for kernel-team@lists.ubuntu.com; Tue, 11 Oct 2022 05:57:14 +0000 Received: from mail-qk1-f199.google.com (mail-qk1-f199.google.com [209.85.222.199]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id 5DA1440275 for ; Tue, 11 Oct 2022 05:57:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1665467834; bh=DnK2GKKmMTEz5t+7U03oq22z0UHJyrK9z6bA+Qt9ItU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=o2g53LqHWrf04TxjsvCHD5cE3uKUpCsX3BBy4GnqIbfreBig311+TAXFDg1qUNTPq 27LmleTMqkKoZX0BSuTE0A/55EAz0JReUqRxbIgU4ojR29OvffMcZZWkFFceopZbRk CruJR+r/aBTanIQqIWCKJ7hZvGTnuQlk9K70PHDMcbWrrI2hCsmJmGqH/yC6GsFGPV jlE8Xm3KNBZcS3bUxvxXMNXk0zOGfx7ARoodoISSAkfqTJrVpIe87YQo1VHTg63WIJ fpLiNCGk2AeOd5yTxzRC2Eu8JLBoL+2WsGrlT6tXhTW6VTMmJDcgNaJRJAgk5dw3Vl 06Ax3MO/u8pJg== Received: by mail-qk1-f199.google.com with SMTP id h7-20020a05620a400700b006cebec84734so11009921qko.23 for ; Mon, 10 Oct 2022 22:57:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DnK2GKKmMTEz5t+7U03oq22z0UHJyrK9z6bA+Qt9ItU=; b=lucUcD0zhPYSXwCq6CTqAo2R/b4l/1jRR49CuA8zyP28es5cY1PAH0dmx38ie1wiLo 5u7mxZh6wLBMaD/+4T7ZsZ7vU9OrZUEwNL1+dyBx4BO1cBeEcyWhTBKQHWiXvRgA6IXO cU8VifgU+pvyNNXGCHBkgGbmiL1pr9W3cQLS+3tjmym9SZBO5FS5ZyZR87qrfiQQ+WLF d2J/4HOz1b+/9CZQITbY3eKvJ2gjULyiRholxGM9gBCiLF2NA4oWvb62BNb9ti86PfY/ O0hJxbh5qxbudj1yJbiVgVP0kYcU+CewGBH3/c8ZYci2jgMP0OT3dEsSkXliPnzcC6Xu kGTg== X-Gm-Message-State: ACrzQf38C6PSOrsP/aXJQJ68Z2z5kVe6BVlSPk9s2suE2U7xbBOD2CfC HSn36aeEXIq9SS/HZJ7QnRVpr1dIyI7z4sW79iNhRRX4Tcn5rMISEpLMx3OcdgLk1CkosyziJp0 /nfs50jZgnqotFVMDLXe/prAvrDA2aVyNegi5WbF9Uw== X-Received: by 2002:ac8:7f14:0:b0:35c:c557:df78 with SMTP id f20-20020ac87f14000000b0035cc557df78mr17479785qtk.209.1665467833454; Mon, 10 Oct 2022 22:57:13 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7yz0NqecwPB2Py7G2fYYP/0ZbotmHVUsz95TC99SeSk/1vJcp2MT6AHvaLjfSFbT+VO15/zw== X-Received: by 2002:ac8:7f14:0:b0:35c:c557:df78 with SMTP id f20-20020ac87f14000000b0035cc557df78mr17479779qtk.209.1665467833041; Mon, 10 Oct 2022 22:57:13 -0700 (PDT) Received: from rpi4-work.fuzzbuzz.org ([38.147.253.164]) by smtp.gmail.com with ESMTPSA id z8-20020ac81008000000b0039351b26714sm9848927qti.7.2022.10.10.22.57.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Oct 2022 22:57:12 -0700 (PDT) From: Khalid Elmously To: kernel-team@lists.ubuntu.com Subject: [PATCH 06/19] gve: Add tx|rx-coalesce-usec for DQO Date: Tue, 11 Oct 2022 01:56:51 -0400 Message-Id: <20221011055704.642271-7-khalid.elmously@canonical.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221011055704.642271-1-khalid.elmously@canonical.com> References: <20221011055704.642271-1-khalid.elmously@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Tao Liu BugLink: https://bugs.launchpad.net/bugs/1953575 Adding ethtool support for changing rx-coalesce-usec and tx-coalesce-usec when using the DQO queue format. Signed-off-by: Tao Liu Signed-off-by: Jeroen de Borst Signed-off-by: David S. Miller (cherry picked from commit 6081ac2013ab9f1a8743f90ef13d973888359cda) Signed-off-by: Khalid Elmously --- drivers/net/ethernet/google/gve/gve.h | 4 ++ drivers/net/ethernet/google/gve/gve_dqo.h | 22 +++++-- drivers/net/ethernet/google/gve/gve_ethtool.c | 61 +++++++++++++++++++ drivers/net/ethernet/google/gve/gve_main.c | 15 +++-- 4 files changed, 91 insertions(+), 11 deletions(-) diff --git a/drivers/net/ethernet/google/gve/gve.h b/drivers/net/ethernet/google/gve/gve.h index d47ef01c17e8..4a1fb0f1a59d 100644 --- a/drivers/net/ethernet/google/gve/gve.h +++ b/drivers/net/ethernet/google/gve/gve.h @@ -568,6 +568,10 @@ struct gve_priv { int data_buffer_size_dqo; enum gve_queue_format queue_format; + + /* Interrupt coalescing settings */ + u32 tx_coalesce_usecs; + u32 rx_coalesce_usecs; }; enum gve_service_task_flags_bit { diff --git a/drivers/net/ethernet/google/gve/gve_dqo.h b/drivers/net/ethernet/google/gve/gve_dqo.h index 836042364124..91523b568e1e 100644 --- a/drivers/net/ethernet/google/gve/gve_dqo.h +++ b/drivers/net/ethernet/google/gve/gve_dqo.h @@ -18,6 +18,7 @@ #define GVE_TX_IRQ_RATELIMIT_US_DQO 50 #define GVE_RX_IRQ_RATELIMIT_US_DQO 20 +#define GVE_MAX_ITR_INTERVAL_DQO (GVE_ITR_INTERVAL_DQO_MASK * 2) /* Timeout in seconds to wait for a reinjection completion after receiving * its corresponding miss completion. @@ -54,17 +55,17 @@ gve_tx_put_doorbell_dqo(const struct gve_priv *priv, } /* Builds register value to write to DQO IRQ doorbell to enable with specified - * ratelimit. + * ITR interval. */ -static inline u32 gve_set_itr_ratelimit_dqo(u32 ratelimit_us) +static inline u32 gve_setup_itr_interval_dqo(u32 interval_us) { u32 result = GVE_ITR_ENABLE_BIT_DQO; /* Interval has 2us granularity. */ - ratelimit_us >>= 1; + interval_us >>= 1; - ratelimit_us &= GVE_ITR_INTERVAL_DQO_MASK; - result |= (ratelimit_us << GVE_ITR_INTERVAL_DQO_SHIFT); + interval_us &= GVE_ITR_INTERVAL_DQO_MASK; + result |= (interval_us << GVE_ITR_INTERVAL_DQO_SHIFT); return result; } @@ -78,4 +79,15 @@ gve_write_irq_doorbell_dqo(const struct gve_priv *priv, iowrite32(val, &priv->db_bar2[index]); } +/* Sets interrupt throttling interval and enables interrupt + * by writing to IRQ doorbell. + */ +static inline void +gve_set_itr_coalesce_usecs_dqo(struct gve_priv *priv, + struct gve_notify_block *block, + u32 usecs) +{ + gve_write_irq_doorbell_dqo(priv, block, + gve_setup_itr_interval_dqo(usecs)); +} #endif /* _GVE_DQO_H_ */ diff --git a/drivers/net/ethernet/google/gve/gve_ethtool.c b/drivers/net/ethernet/google/gve/gve_ethtool.c index 716e6240305d..aefa47096693 100644 --- a/drivers/net/ethernet/google/gve/gve_ethtool.c +++ b/drivers/net/ethernet/google/gve/gve_ethtool.c @@ -8,6 +8,7 @@ #include #include "gve.h" #include "gve_adminq.h" +#include "gve_dqo.h" static void gve_get_drvinfo(struct net_device *netdev, struct ethtool_drvinfo *info) @@ -532,7 +533,65 @@ static int gve_get_link_ksettings(struct net_device *netdev, return err; } +static int gve_get_coalesce(struct net_device *netdev, + struct ethtool_coalesce *ec, + struct kernel_ethtool_coalesce *kernel_ec, + struct netlink_ext_ack *extack) +{ + struct gve_priv *priv = netdev_priv(netdev); + + if (gve_is_gqi(priv)) + return -EOPNOTSUPP; + ec->tx_coalesce_usecs = priv->tx_coalesce_usecs; + ec->rx_coalesce_usecs = priv->rx_coalesce_usecs; + + return 0; +} + +static int gve_set_coalesce(struct net_device *netdev, + struct ethtool_coalesce *ec, + struct kernel_ethtool_coalesce *kernel_ec, + struct netlink_ext_ack *extack) +{ + struct gve_priv *priv = netdev_priv(netdev); + u32 tx_usecs_orig = priv->tx_coalesce_usecs; + u32 rx_usecs_orig = priv->rx_coalesce_usecs; + int idx; + + if (gve_is_gqi(priv)) + return -EOPNOTSUPP; + + if (ec->tx_coalesce_usecs > GVE_MAX_ITR_INTERVAL_DQO || + ec->rx_coalesce_usecs > GVE_MAX_ITR_INTERVAL_DQO) + return -EINVAL; + priv->tx_coalesce_usecs = ec->tx_coalesce_usecs; + priv->rx_coalesce_usecs = ec->rx_coalesce_usecs; + + if (tx_usecs_orig != priv->tx_coalesce_usecs) { + for (idx = 0; idx < priv->tx_cfg.num_queues; idx++) { + int ntfy_idx = gve_tx_idx_to_ntfy(priv, idx); + struct gve_notify_block *block = &priv->ntfy_blocks[ntfy_idx]; + + gve_set_itr_coalesce_usecs_dqo(priv, block, + priv->tx_coalesce_usecs); + } + } + + if (rx_usecs_orig != priv->rx_coalesce_usecs) { + for (idx = 0; idx < priv->rx_cfg.num_queues; idx++) { + int ntfy_idx = gve_rx_idx_to_ntfy(priv, idx); + struct gve_notify_block *block = &priv->ntfy_blocks[ntfy_idx]; + + gve_set_itr_coalesce_usecs_dqo(priv, block, + priv->rx_coalesce_usecs); + } + } + + return 0; +} + const struct ethtool_ops gve_ethtool_ops = { + .supported_coalesce_params = ETHTOOL_COALESCE_USECS, .get_drvinfo = gve_get_drvinfo, .get_strings = gve_get_strings, .get_sset_count = gve_get_sset_count, @@ -542,6 +601,8 @@ const struct ethtool_ops gve_ethtool_ops = { .set_channels = gve_set_channels, .get_channels = gve_get_channels, .get_link = ethtool_op_get_link, + .get_coalesce = gve_get_coalesce, + .set_coalesce = gve_set_coalesce, .get_ringparam = gve_get_ringparam, .reset = gve_user_reset, .get_tunable = gve_get_tunable, diff --git a/drivers/net/ethernet/google/gve/gve_main.c b/drivers/net/ethernet/google/gve/gve_main.c index 68552848d388..d12cfd721a02 100644 --- a/drivers/net/ethernet/google/gve/gve_main.c +++ b/drivers/net/ethernet/google/gve/gve_main.c @@ -1095,9 +1095,8 @@ static void gve_turnup(struct gve_priv *priv) if (gve_is_gqi(priv)) { iowrite32be(0, gve_irq_doorbell(priv, block)); } else { - u32 val = gve_set_itr_ratelimit_dqo(GVE_TX_IRQ_RATELIMIT_US_DQO); - - gve_write_irq_doorbell_dqo(priv, block, val); + gve_set_itr_coalesce_usecs_dqo(priv, block, + priv->tx_coalesce_usecs); } } for (idx = 0; idx < priv->rx_cfg.num_queues; idx++) { @@ -1108,9 +1107,8 @@ static void gve_turnup(struct gve_priv *priv) if (gve_is_gqi(priv)) { iowrite32be(0, gve_irq_doorbell(priv, block)); } else { - u32 val = gve_set_itr_ratelimit_dqo(GVE_RX_IRQ_RATELIMIT_US_DQO); - - gve_write_irq_doorbell_dqo(priv, block, val); + gve_set_itr_coalesce_usecs_dqo(priv, block, + priv->rx_coalesce_usecs); } } @@ -1415,6 +1413,11 @@ static int gve_init_priv(struct gve_priv *priv, bool skip_describe_device) dev_info(&priv->pdev->dev, "Max TX queues %d, Max RX queues %d\n", priv->tx_cfg.max_queues, priv->rx_cfg.max_queues); + if (!gve_is_gqi(priv)) { + priv->tx_coalesce_usecs = GVE_TX_IRQ_RATELIMIT_US_DQO; + priv->rx_coalesce_usecs = GVE_RX_IRQ_RATELIMIT_US_DQO; + } + setup_device: err = gve_setup_device_resources(priv); if (!err)