From patchwork Tue Sep 1 21:51:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Awogbemila X-Patchwork-Id: 1355416 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20161025 header.b=hB3MqYsB; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4Bh14k1h29z9sR4 for ; Wed, 2 Sep 2020 07:52:06 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729305AbgIAVwF (ORCPT ); Tue, 1 Sep 2020 17:52:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56764 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727833AbgIAVwB (ORCPT ); Tue, 1 Sep 2020 17:52:01 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9E88DC061245 for ; Tue, 1 Sep 2020 14:52:00 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id x10so2663955ybj.19 for ; Tue, 01 Sep 2020 14:52:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=YBDBTYJyAAyHSIMfe7u64IQoZsgba0wVm0w4On0MyKI=; b=hB3MqYsBs7cw/1Ipshxwo2lyNYatyRkdhq/cgNhag9zfjGsHTfzQVndgM7pF/RHhE7 BZ/yHS19zCQJ8hXCk6m6mfl55T/GQClu56RJtzi+y5l3i6rEjeV8+8lh84iWXcvo1YaZ RYhxvYyFYogxqRAmlIbzpR9u3XNK0bqGRZ4B0ikukCzQ2sbxrzulflng/KZq7ioDycZM xfdYd97akpmbmWkPAm1jIJ7cY/GMyWOZvh7HJzlsdDxvsQ1ls1NAf9g+XT+pl4nTQgzd UKI1fryC51bRCA5l9FSRjbVDDZCx18N6LxtbF9TMrQlQ7dc+enad15zwqKYX5PddhIf1 YUkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=YBDBTYJyAAyHSIMfe7u64IQoZsgba0wVm0w4On0MyKI=; b=pGtMd91MSO2zD6pAf6ESQFnpSc+1GtkdQZAKxMWLAAJadv1Ru3We7ZWyyJk4CZAqQN 0xgALGT8yjjOJ6O080mQAFuZLuwyZTSJ2Wng/KGpJGAQ2araqlM+LZuEynZpSeX2tku/ Jqji+mD79FjnCnwyw4tIVzC8Juc3zM597+9CIeYNDYMN6Xlp+2lM4mqJL4hS0ooYKU+i Fv0+48MX0xV3KcBgrZrh7KIOnJrOKt6uPOTud0EMV7/JenbIvNN2wY+wSeOCml9LBph9 xgkhDbJQ0PvJvpdMPrk8JL1T6mb/4yAkbUL1FJPmeGoOlsmAMS/AuDUGxf5c2CIWoFij vkkQ== X-Gm-Message-State: AOAM533cEEzQhg+Aj3N7dMNpdBgmihJBE9/DjzfrrQFJWwe7NSkQ7Trn kGyPbTc9/Iu8ZcvQgmoJM1W5mMA+mFF9edDpumeQkEfgpobZOsI8nPFOdF3CgsN9+cFmPt9dD0+ uxScZLkn8bwwSj1XUKDLYtioH+WZ3TCauO3GPKi8aFj9vmEt7H9r0O8cJk/+nhOOmkuS0DQQX X-Google-Smtp-Source: ABdhPJyf4yi7uQ4xUkuRMAoPEU+A59shwoVs9toIGPiSMbCERQemo5tki7J0Ngsh5KLxzvjL/+rlOiWnaFF18jxv X-Received: from awogbemila.sea.corp.google.com ([2620:15c:100:202:1ea0:b8ff:fe73:6cc0]) (user=awogbemila job=sendgmr) by 2002:a25:5755:: with SMTP id l82mr6008189ybb.175.1598997118828; Tue, 01 Sep 2020 14:51:58 -0700 (PDT) Date: Tue, 1 Sep 2020 14:51:41 -0700 In-Reply-To: <20200901215149.2685117-1-awogbemila@google.com> Message-Id: <20200901215149.2685117-2-awogbemila@google.com> Mime-Version: 1.0 References: <20200901215149.2685117-1-awogbemila@google.com> X-Mailer: git-send-email 2.28.0.402.g5ffc5be6b7-goog Subject: [PATCH net-next v2 1/9] gve: Get and set Rx copybreak via ethtool From: David Awogbemila To: netdev@vger.kernel.org Cc: Kuo Zhao , Yangchun Fu , David Awogbemila Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Kuo Zhao This adds support for getting and setting the RX copybreak value via ethtool. Reviewed-by: Yangchun Fu Signed-off-by: Kuo Zhao Signed-off-by: David Awogbemila Reviewed-by: Andrew Lunn --- drivers/net/ethernet/google/gve/gve_ethtool.c | 34 +++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/drivers/net/ethernet/google/gve/gve_ethtool.c b/drivers/net/ethernet/google/gve/gve_ethtool.c index d8fa816f4473..1a80d38e66ec 100644 --- a/drivers/net/ethernet/google/gve/gve_ethtool.c +++ b/drivers/net/ethernet/google/gve/gve_ethtool.c @@ -230,6 +230,38 @@ static int gve_user_reset(struct net_device *netdev, u32 *flags) return -EOPNOTSUPP; } +static int gve_get_tunable(struct net_device *netdev, + const struct ethtool_tunable *etuna, void *value) +{ + struct gve_priv *priv = netdev_priv(netdev); + + switch (etuna->id) { + case ETHTOOL_RX_COPYBREAK: + *(u32 *)value = priv->rx_copybreak; + return 0; + default: + return -EOPNOTSUPP; + } +} + +static int gve_set_tunable(struct net_device *netdev, + const struct ethtool_tunable *etuna, const void *value) +{ + struct gve_priv *priv = netdev_priv(netdev); + u32 len; + + switch (etuna->id) { + case ETHTOOL_RX_COPYBREAK: + len = *(u32 *)value; + if (len > PAGE_SIZE / 2) + return -EINVAL; + priv->rx_copybreak = len; + return 0; + default: + return -EOPNOTSUPP; + } +} + const struct ethtool_ops gve_ethtool_ops = { .get_drvinfo = gve_get_drvinfo, .get_strings = gve_get_strings, @@ -242,4 +274,6 @@ const struct ethtool_ops gve_ethtool_ops = { .get_link = ethtool_op_get_link, .get_ringparam = gve_get_ringparam, .reset = gve_user_reset, + .get_tunable = gve_get_tunable, + .set_tunable = gve_set_tunable, }; From patchwork Tue Sep 1 21:51:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Awogbemila X-Patchwork-Id: 1355417 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20161025 header.b=MiBfy4Mc; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4Bh14l6cpNz9sR4 for ; Wed, 2 Sep 2020 07:52:07 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729478AbgIAVwG (ORCPT ); Tue, 1 Sep 2020 17:52:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56766 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727946AbgIAVwB (ORCPT ); Tue, 1 Sep 2020 17:52:01 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 73738C061246 for ; Tue, 1 Sep 2020 14:52:01 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id v11so2681069ybm.22 for ; Tue, 01 Sep 2020 14:52:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=lxB3dDii8QkaIwUakotPkYx0le0Zs8cTAJF1yBNkccM=; b=MiBfy4Mcw4wELj+5kRHWM/+Jqzjfh7z3vzzz/Ez/FG2wiPo24PEDkhF9dAtvsX+EFj enS/SsfDI+3Tn6LpzB4NUPOGJwE8vVrgnBfwWsrz7DcXok5BDpIE6HDvQbfpx/+ZGp1R xcuVJOKsSz8yNQ9fM3ox51ck5w2SCF6xRfk9W8fEmytVkv14qOWt5DSMvpbTpdnJ1G82 59b8N8EgFeF2Af+O6kJ3zhjgiZdv21RZsqmgwgPdzu0WvK3Tw0wyfiAYGyTnu6Xdh/rB gtVep8hxpKW8IqEPHW35HmAuwmskUEvxil1ZqKtJbr3ednmTM6d3wVkpe8lktgapZTAK XpYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=lxB3dDii8QkaIwUakotPkYx0le0Zs8cTAJF1yBNkccM=; b=BKVQdpS6X9TWkKF1sUEvS3GIj/AFkr/Wo4HEpsHauV/2JPtrHXNnHqR6pD5B5URvX0 ggevuZ831mQdXw61wSuWNf6beKf/Zfp/n0Pco+5jK85HguLh47UcwodZ63LwjkQ4SlzM 6iZFol9o22qyhaInXZISolXz2kMeuIU3ZJUNq8Q1XcfNgEOx2yCJmNybPpY2qYh1M/WM Lbiw1AEQm76VPSKmTJIzC5JksOrc+4OxRzMmSa1YV/18/fSmAPMA+cUt/Gc8o2FXILIJ UkaIdfBO08QjkH+LW2ECZgDYunUm3atBxU7UZ9ac5SxaFJF4X2L1Cofo5ZrN/rrEfBhV O+BQ== X-Gm-Message-State: AOAM530DX5gSVJtKSu24EgmT3tLYYOsNI4IZkSyxoHmttPA+VlA+lisq Ckmn6AxvY9L/x+1bCQ66oSaXBkUn+T3rv8vENUe+SG6bRV9vT79W9HZJVW7VBfXC5t6jvHkgWG4 LWJU23rVWiwDknekJEwnZDrFFpHtbdm+6sHYW9KIksgip5UZTbsl1U1IXAmbWcHiRViNsMQrE X-Google-Smtp-Source: ABdhPJxTF7pD6oulzcubv2GDbDUoOz2Eippi/KmGgD1V/4BckjMOWpm4Xn0XyVJFfRNMVVbhMUm3Wolp7xsqKfgz X-Received: from awogbemila.sea.corp.google.com ([2620:15c:100:202:1ea0:b8ff:fe73:6cc0]) (user=awogbemila job=sendgmr) by 2002:a25:fc06:: with SMTP id v6mr6037646ybd.478.1598997120538; Tue, 01 Sep 2020 14:52:00 -0700 (PDT) Date: Tue, 1 Sep 2020 14:51:42 -0700 In-Reply-To: <20200901215149.2685117-1-awogbemila@google.com> Message-Id: <20200901215149.2685117-3-awogbemila@google.com> Mime-Version: 1.0 References: <20200901215149.2685117-1-awogbemila@google.com> X-Mailer: git-send-email 2.28.0.402.g5ffc5be6b7-goog Subject: [PATCH net-next v2 2/9] gve: Add stats for gve. From: David Awogbemila To: netdev@vger.kernel.org Cc: Kuo Zhao , Yangchun Fu , David Awogbemila Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Kuo Zhao Sample output of "ethtool -S " with 1 RX queue and 1 TX queue: NIC statistics: rx_packets: 1039 tx_packets: 37 rx_bytes: 822071 tx_bytes: 4100 rx_dropped: 0 tx_dropped: 0 tx_timeouts: 0 rx_skb_alloc_fail: 0 rx_buf_alloc_fail: 0 rx_desc_err_dropped_pkt: 0 interface_up_cnt: 1 interface_down_cnt: 0 reset_cnt: 0 page_alloc_fail: 0 dma_mapping_error: 0 rx_posted_desc[0]: 1365 rx_completed_desc[0]: 341 rx_bytes[0]: 215094 rx_dropped_pkt[0]: 0 rx_copybreak_pkt[0]: 3 rx_copied_pkt[0]: 3 tx_posted_desc[0]: 6 tx_completed_desc[0]: 6 tx_bytes[0]: 420 tx_wake[0]: 0 tx_stop[0]: 0 tx_event_counter[0]: 6 adminq_prod_cnt: 34 adminq_cmd_fail: 0 adminq_timeouts: 0 adminq_describe_device_cnt: 1 adminq_cfg_device_resources_cnt: 1 adminq_register_page_list_cnt: 16 adminq_unregister_page_list_cnt: 0 adminq_create_tx_queue_cnt: 8 adminq_create_rx_queue_cnt: 8 adminq_destroy_tx_queue_cnt: 0 adminq_destroy_rx_queue_cnt: 0 adminq_dcfg_device_resources_cnt: 0 adminq_set_driver_parameter_cnt: 0 Reviewed-by: Yangchun Fu Signed-off-by: Kuo Zhao Signed-off-by: David Awogbemila Acked-by: Jakub Kicinski --- drivers/net/ethernet/google/gve/gve.h | 28 +++- drivers/net/ethernet/google/gve/gve_adminq.c | 65 +++++++- drivers/net/ethernet/google/gve/gve_ethtool.c | 149 ++++++++++++++---- drivers/net/ethernet/google/gve/gve_main.c | 15 +- drivers/net/ethernet/google/gve/gve_rx.c | 37 ++++- 5 files changed, 246 insertions(+), 48 deletions(-) diff --git a/drivers/net/ethernet/google/gve/gve.h b/drivers/net/ethernet/google/gve/gve.h index ebc37e256922..55b34b437764 100644 --- a/drivers/net/ethernet/google/gve/gve.h +++ b/drivers/net/ethernet/google/gve/gve.h @@ -71,6 +71,11 @@ struct gve_rx_ring { u32 cnt; /* free-running total number of completed packets */ u32 fill_cnt; /* free-running total number of descs and buffs posted */ u32 mask; /* masks the cnt and fill_cnt to the size of the ring */ + u64 rx_copybreak_pkt; /* free-running count of copybreak packets */ + u64 rx_copied_pkt; /* free-running total number of copied packets */ + u64 rx_skb_alloc_fail; /* free-running count of skb alloc fails */ + u64 rx_buf_alloc_fail; /* free-running count of buffer alloc fails */ + u64 rx_desc_err_dropped_pkt; /* free-running count of packets dropped by descriptor error */ u32 q_num; /* queue index */ u32 ntfy_id; /* notification block index */ struct gve_queue_resources *q_resources; /* head and tail pointer idx */ @@ -202,6 +207,26 @@ struct gve_priv { dma_addr_t adminq_bus_addr; u32 adminq_mask; /* masks prod_cnt to adminq size */ u32 adminq_prod_cnt; /* free-running count of AQ cmds executed */ + u32 adminq_cmd_fail; /* free-running count of AQ cmds failed */ + u32 adminq_timeouts; /* free-running count of AQ cmds timeouts */ + /* free-running count of per AQ cmd executed */ + u32 adminq_describe_device_cnt; + u32 adminq_cfg_device_resources_cnt; + u32 adminq_register_page_list_cnt; + u32 adminq_unregister_page_list_cnt; + u32 adminq_create_tx_queue_cnt; + u32 adminq_create_rx_queue_cnt; + u32 adminq_destroy_tx_queue_cnt; + u32 adminq_destroy_rx_queue_cnt; + u32 adminq_dcfg_device_resources_cnt; + u32 adminq_set_driver_parameter_cnt; + + /* Global stats */ + u32 interface_up_cnt; /* count of times interface turned up since last reset */ + u32 interface_down_cnt; /* count of times interface turned down since last reset */ + u32 reset_cnt; /* count of reset */ + u32 page_alloc_fail; /* count of page alloc fails */ + u32 dma_mapping_error; /* count of dma mapping errors */ struct workqueue_struct *gve_wq; struct work_struct service_task; @@ -426,7 +451,8 @@ static inline bool gve_can_recycle_pages(struct net_device *dev) } /* buffers */ -int gve_alloc_page(struct device *dev, struct page **page, dma_addr_t *dma, +int gve_alloc_page(struct gve_priv *priv, struct device *dev, + struct page **page, dma_addr_t *dma, enum dma_data_direction); void gve_free_page(struct device *dev, struct page *page, dma_addr_t dma, enum dma_data_direction); diff --git a/drivers/net/ethernet/google/gve/gve_adminq.c b/drivers/net/ethernet/google/gve/gve_adminq.c index c3ba7baf0107..529de756ff9b 100644 --- a/drivers/net/ethernet/google/gve/gve_adminq.c +++ b/drivers/net/ethernet/google/gve/gve_adminq.c @@ -23,6 +23,18 @@ int gve_adminq_alloc(struct device *dev, struct gve_priv *priv) priv->adminq_mask = (PAGE_SIZE / sizeof(union gve_adminq_command)) - 1; priv->adminq_prod_cnt = 0; + priv->adminq_cmd_fail = 0; + priv->adminq_timeouts = 0; + priv->adminq_describe_device_cnt = 0; + priv->adminq_cfg_device_resources_cnt = 0; + priv->adminq_register_page_list_cnt = 0; + priv->adminq_unregister_page_list_cnt = 0; + priv->adminq_create_tx_queue_cnt = 0; + priv->adminq_create_rx_queue_cnt = 0; + priv->adminq_destroy_tx_queue_cnt = 0; + priv->adminq_destroy_rx_queue_cnt = 0; + priv->adminq_dcfg_device_resources_cnt = 0; + priv->adminq_set_driver_parameter_cnt = 0; /* Setup Admin queue with the device */ iowrite32be(priv->adminq_bus_addr / PAGE_SIZE, @@ -81,17 +93,18 @@ static bool gve_adminq_wait_for_cmd(struct gve_priv *priv, u32 prod_cnt) return false; } -static int gve_adminq_parse_err(struct device *dev, u32 status) +static int gve_adminq_parse_err(struct gve_priv *priv, u32 status) { if (status != GVE_ADMINQ_COMMAND_PASSED && - status != GVE_ADMINQ_COMMAND_UNSET) - dev_err(dev, "AQ command failed with status %d\n", status); - + status != GVE_ADMINQ_COMMAND_UNSET) { + dev_err(&priv->pdev->dev, "AQ command failed with status %d\n", status); + priv->adminq_cmd_fail++; + } switch (status) { case GVE_ADMINQ_COMMAND_PASSED: return 0; case GVE_ADMINQ_COMMAND_UNSET: - dev_err(dev, "parse_aq_err: err and status both unset, this should not be possible.\n"); + dev_err(&priv->pdev->dev, "parse_aq_err: err and status both unset, this should not be possible.\n"); return -EINVAL; case GVE_ADMINQ_COMMAND_ERROR_ABORTED: case GVE_ADMINQ_COMMAND_ERROR_CANCELLED: @@ -116,7 +129,7 @@ static int gve_adminq_parse_err(struct device *dev, u32 status) case GVE_ADMINQ_COMMAND_ERROR_UNIMPLEMENTED: return -ENOTSUPP; default: - dev_err(dev, "parse_aq_err: unknown status code %d\n", status); + dev_err(&priv->pdev->dev, "parse_aq_err: unknown status code %d\n", status); return -EINVAL; } } @@ -130,22 +143,60 @@ int gve_adminq_execute_cmd(struct gve_priv *priv, union gve_adminq_command *cmd; u32 status = 0; u32 prod_cnt; + u32 opcode; cmd = &priv->adminq[priv->adminq_prod_cnt & priv->adminq_mask]; priv->adminq_prod_cnt++; prod_cnt = priv->adminq_prod_cnt; memcpy(cmd, cmd_orig, sizeof(*cmd_orig)); + opcode = be32_to_cpu(READ_ONCE(cmd->opcode)); + + switch (opcode) { + case GVE_ADMINQ_DESCRIBE_DEVICE: + priv->adminq_describe_device_cnt++; + break; + case GVE_ADMINQ_CONFIGURE_DEVICE_RESOURCES: + priv->adminq_cfg_device_resources_cnt++; + break; + case GVE_ADMINQ_REGISTER_PAGE_LIST: + priv->adminq_register_page_list_cnt++; + break; + case GVE_ADMINQ_UNREGISTER_PAGE_LIST: + priv->adminq_unregister_page_list_cnt++; + break; + case GVE_ADMINQ_CREATE_TX_QUEUE: + priv->adminq_create_tx_queue_cnt++; + break; + case GVE_ADMINQ_CREATE_RX_QUEUE: + priv->adminq_create_rx_queue_cnt++; + break; + case GVE_ADMINQ_DESTROY_TX_QUEUE: + priv->adminq_destroy_tx_queue_cnt++; + break; + case GVE_ADMINQ_DESTROY_RX_QUEUE: + priv->adminq_destroy_rx_queue_cnt++; + break; + case GVE_ADMINQ_DECONFIGURE_DEVICE_RESOURCES: + priv->adminq_dcfg_device_resources_cnt++; + break; + case GVE_ADMINQ_SET_DRIVER_PARAMETER: + priv->adminq_set_driver_parameter_cnt++; + break; + default: + dev_err(&priv->pdev->dev, "unknown AQ command opcode %d\n", opcode); + } gve_adminq_kick_cmd(priv, prod_cnt); if (!gve_adminq_wait_for_cmd(priv, prod_cnt)) { dev_err(&priv->pdev->dev, "AQ command timed out, need to reset AQ\n"); + priv->adminq_timeouts++; return -ENOTRECOVERABLE; } memcpy(cmd_orig, cmd, sizeof(*cmd)); status = be32_to_cpu(READ_ONCE(cmd->status)); - return gve_adminq_parse_err(&priv->pdev->dev, status); + return gve_adminq_parse_err(priv, status); } /* The device specifies that the management vector can either be the first irq diff --git a/drivers/net/ethernet/google/gve/gve_ethtool.c b/drivers/net/ethernet/google/gve/gve_ethtool.c index 1a80d38e66ec..28d831d52701 100644 --- a/drivers/net/ethernet/google/gve/gve_ethtool.c +++ b/drivers/net/ethernet/google/gve/gve_ethtool.c @@ -34,17 +34,40 @@ static u32 gve_get_msglevel(struct net_device *netdev) static const char gve_gstrings_main_stats[][ETH_GSTRING_LEN] = { "rx_packets", "tx_packets", "rx_bytes", "tx_bytes", "rx_dropped", "tx_dropped", "tx_timeouts", + "rx_skb_alloc_fail", "rx_buf_alloc_fail", "rx_desc_err_dropped_pkt", + "interface_up_cnt", "interface_down_cnt", "reset_cnt", + "page_alloc_fail", "dma_mapping_error", +}; + +static const char gve_gstrings_rx_stats[][ETH_GSTRING_LEN] = { + "rx_posted_desc[%u]", "rx_completed_desc[%u]", "rx_bytes[%u]", + "rx_dropped_pkt[%u]", "rx_copybreak_pkt[%u]", "rx_copied_pkt[%u]", +}; + +static const char gve_gstrings_tx_stats[][ETH_GSTRING_LEN] = { + "tx_posted_desc[%u]", "tx_completed_desc[%u]", "tx_bytes[%u]", + "tx_wake[%u]", "tx_stop[%u]", "tx_event_counter[%u]", +}; + +static const char gve_gstrings_adminq_stats[][ETH_GSTRING_LEN] = { + "adminq_prod_cnt", "adminq_cmd_fail", "adminq_timeouts", + "adminq_describe_device_cnt", "adminq_cfg_device_resources_cnt", + "adminq_register_page_list_cnt", "adminq_unregister_page_list_cnt", + "adminq_create_tx_queue_cnt", "adminq_create_rx_queue_cnt", + "adminq_destroy_tx_queue_cnt", "adminq_destroy_rx_queue_cnt", + "adminq_dcfg_device_resources_cnt", "adminq_set_driver_parameter_cnt", }; #define GVE_MAIN_STATS_LEN ARRAY_SIZE(gve_gstrings_main_stats) -#define NUM_GVE_TX_CNTS 5 -#define NUM_GVE_RX_CNTS 2 +#define GVE_ADMINQ_STATS_LEN ARRAY_SIZE(gve_gstrings_adminq_stats) +#define NUM_GVE_TX_CNTS ARRAY_SIZE(gve_gstrings_tx_stats) +#define NUM_GVE_RX_CNTS ARRAY_SIZE(gve_gstrings_rx_stats) static void gve_get_strings(struct net_device *netdev, u32 stringset, u8 *data) { struct gve_priv *priv = netdev_priv(netdev); char *s = (char *)data; - int i; + int i, j; if (stringset != ETH_SS_STATS) return; @@ -52,24 +75,24 @@ static void gve_get_strings(struct net_device *netdev, u32 stringset, u8 *data) memcpy(s, *gve_gstrings_main_stats, sizeof(gve_gstrings_main_stats)); s += sizeof(gve_gstrings_main_stats); + for (i = 0; i < priv->rx_cfg.num_queues; i++) { - snprintf(s, ETH_GSTRING_LEN, "rx_desc_cnt[%u]", i); - s += ETH_GSTRING_LEN; - snprintf(s, ETH_GSTRING_LEN, "rx_desc_fill_cnt[%u]", i); - s += ETH_GSTRING_LEN; + for (j = 0; j < NUM_GVE_RX_CNTS; j++) { + snprintf(s, ETH_GSTRING_LEN, gve_gstrings_rx_stats[j], i); + s += ETH_GSTRING_LEN; + } } + for (i = 0; i < priv->tx_cfg.num_queues; i++) { - snprintf(s, ETH_GSTRING_LEN, "tx_req[%u]", i); - s += ETH_GSTRING_LEN; - snprintf(s, ETH_GSTRING_LEN, "tx_done[%u]", i); - s += ETH_GSTRING_LEN; - snprintf(s, ETH_GSTRING_LEN, "tx_wake[%u]", i); - s += ETH_GSTRING_LEN; - snprintf(s, ETH_GSTRING_LEN, "tx_stop[%u]", i); - s += ETH_GSTRING_LEN; - snprintf(s, ETH_GSTRING_LEN, "tx_event_counter[%u]", i); - s += ETH_GSTRING_LEN; + for (j = 0; j < NUM_GVE_TX_CNTS; j++) { + snprintf(s, ETH_GSTRING_LEN, gve_gstrings_tx_stats[j], i); + s += ETH_GSTRING_LEN; + } } + + memcpy(s, *gve_gstrings_adminq_stats, + sizeof(gve_gstrings_adminq_stats)); + s += sizeof(gve_gstrings_adminq_stats); } static int gve_get_sset_count(struct net_device *netdev, int sset) @@ -78,7 +101,7 @@ static int gve_get_sset_count(struct net_device *netdev, int sset) switch (sset) { case ETH_SS_STATS: - return GVE_MAIN_STATS_LEN + + return GVE_MAIN_STATS_LEN + GVE_ADMINQ_STATS_LEN + (priv->rx_cfg.num_queues * NUM_GVE_RX_CNTS) + (priv->tx_cfg.num_queues * NUM_GVE_TX_CNTS); default: @@ -90,24 +113,39 @@ static void gve_get_ethtool_stats(struct net_device *netdev, struct ethtool_stats *stats, u64 *data) { - struct gve_priv *priv = netdev_priv(netdev); - u64 rx_pkts, rx_bytes, tx_pkts, tx_bytes; + u64 tmp_rx_pkts, tmp_rx_bytes, tmp_rx_skb_alloc_fail, tmp_rx_buf_alloc_fail, + tmp_rx_desc_err_dropped_pkt, tmp_tx_pkts, tmp_tx_bytes; + u64 rx_buf_alloc_fail, rx_desc_err_dropped_pkt, rx_pkts, + rx_skb_alloc_fail, rx_bytes, tx_pkts, tx_bytes; + struct gve_priv *priv; unsigned int start; int ring; int i; ASSERT_RTNL(); - for (rx_pkts = 0, rx_bytes = 0, ring = 0; + priv = netdev_priv(netdev); + for (rx_pkts = 0, rx_bytes = 0, rx_skb_alloc_fail = 0, + rx_buf_alloc_fail = 0, rx_desc_err_dropped_pkt = 0, ring = 0; ring < priv->rx_cfg.num_queues; ring++) { if (priv->rx) { do { + struct gve_rx_ring *rx = &priv->rx[ring]; start = u64_stats_fetch_begin(&priv->rx[ring].statss); - rx_pkts += priv->rx[ring].rpackets; - rx_bytes += priv->rx[ring].rbytes; + tmp_rx_pkts = rx->rpackets; + tmp_rx_bytes = rx->rbytes; + tmp_rx_skb_alloc_fail = rx->rx_skb_alloc_fail; + tmp_rx_buf_alloc_fail = rx->rx_buf_alloc_fail; + tmp_rx_desc_err_dropped_pkt = + rx->rx_desc_err_dropped_pkt; } while (u64_stats_fetch_retry(&priv->rx[ring].statss, start)); + rx_pkts += tmp_rx_pkts; + rx_bytes += tmp_rx_bytes; + rx_skb_alloc_fail += tmp_rx_skb_alloc_fail; + rx_buf_alloc_fail += tmp_rx_buf_alloc_fail; + rx_desc_err_dropped_pkt += tmp_rx_desc_err_dropped_pkt; } } for (tx_pkts = 0, tx_bytes = 0, ring = 0; @@ -116,10 +154,12 @@ gve_get_ethtool_stats(struct net_device *netdev, do { start = u64_stats_fetch_begin(&priv->tx[ring].statss); - tx_pkts += priv->tx[ring].pkt_done; - tx_bytes += priv->tx[ring].bytes_done; + tmp_tx_pkts = priv->tx[ring].pkt_done; + tmp_tx_bytes = priv->tx[ring].bytes_done; } while (u64_stats_fetch_retry(&priv->tx[ring].statss, start)); + tx_pkts += tmp_tx_pkts; + tx_bytes += tmp_tx_bytes; } } @@ -128,9 +168,21 @@ gve_get_ethtool_stats(struct net_device *netdev, data[i++] = tx_pkts; data[i++] = rx_bytes; data[i++] = tx_bytes; - /* Skip rx_dropped and tx_dropped */ - i += 2; + /* total rx dropped packets */ + data[i++] = rx_skb_alloc_fail + rx_buf_alloc_fail + + rx_desc_err_dropped_pkt; + /* Skip tx_dropped */ + i++; + data[i++] = priv->tx_timeo_cnt; + data[i++] = rx_skb_alloc_fail; + data[i++] = rx_buf_alloc_fail; + data[i++] = rx_desc_err_dropped_pkt; + data[i++] = priv->interface_up_cnt; + data[i++] = priv->interface_down_cnt; + data[i++] = priv->reset_cnt; + data[i++] = priv->page_alloc_fail; + data[i++] = priv->dma_mapping_error; i = GVE_MAIN_STATS_LEN; /* walk RX rings */ @@ -138,8 +190,25 @@ gve_get_ethtool_stats(struct net_device *netdev, for (ring = 0; ring < priv->rx_cfg.num_queues; ring++) { struct gve_rx_ring *rx = &priv->rx[ring]; - data[i++] = rx->cnt; data[i++] = rx->fill_cnt; + data[i++] = rx->cnt; + do { + start = + u64_stats_fetch_begin(&priv->rx[ring].statss); + tmp_rx_bytes = rx->rbytes; + tmp_rx_skb_alloc_fail = rx->rx_skb_alloc_fail; + tmp_rx_buf_alloc_fail = rx->rx_buf_alloc_fail; + tmp_rx_desc_err_dropped_pkt = + rx->rx_desc_err_dropped_pkt; + } while (u64_stats_fetch_retry(&priv->rx[ring].statss, + start)); + data[i++] = tmp_rx_bytes; + /* rx dropped packets */ + data[i++] = tmp_rx_skb_alloc_fail + + tmp_rx_buf_alloc_fail + + tmp_rx_desc_err_dropped_pkt; + data[i++] = rx->rx_copybreak_pkt; + data[i++] = rx->rx_copied_pkt; } } else { i += priv->rx_cfg.num_queues * NUM_GVE_RX_CNTS; @@ -151,6 +220,13 @@ gve_get_ethtool_stats(struct net_device *netdev, data[i++] = tx->req; data[i++] = tx->done; + do { + start = + u64_stats_fetch_begin(&priv->tx[ring].statss); + tmp_tx_bytes = tx->bytes_done; + } while (u64_stats_fetch_retry(&priv->tx[ring].statss, + start)); + data[i++] = tmp_tx_bytes; data[i++] = tx->wake_queue; data[i++] = tx->stop_queue; data[i++] = be32_to_cpu(gve_tx_load_event_counter(priv, @@ -159,6 +235,20 @@ gve_get_ethtool_stats(struct net_device *netdev, } else { i += priv->tx_cfg.num_queues * NUM_GVE_TX_CNTS; } + /* AQ Stats */ + data[i++] = priv->adminq_prod_cnt; + data[i++] = priv->adminq_cmd_fail; + data[i++] = priv->adminq_timeouts; + data[i++] = priv->adminq_describe_device_cnt; + data[i++] = priv->adminq_cfg_device_resources_cnt; + data[i++] = priv->adminq_register_page_list_cnt; + data[i++] = priv->adminq_unregister_page_list_cnt; + data[i++] = priv->adminq_create_tx_queue_cnt; + data[i++] = priv->adminq_create_rx_queue_cnt; + data[i++] = priv->adminq_destroy_tx_queue_cnt; + data[i++] = priv->adminq_destroy_rx_queue_cnt; + data[i++] = priv->adminq_dcfg_device_resources_cnt; + data[i++] = priv->adminq_set_driver_parameter_cnt; } static void gve_get_channels(struct net_device *netdev, @@ -245,7 +335,8 @@ static int gve_get_tunable(struct net_device *netdev, } static int gve_set_tunable(struct net_device *netdev, - const struct ethtool_tunable *etuna, const void *value) + const struct ethtool_tunable *etuna, + const void *value) { struct gve_priv *priv = netdev_priv(netdev); u32 len; diff --git a/drivers/net/ethernet/google/gve/gve_main.c b/drivers/net/ethernet/google/gve/gve_main.c index e032563ceefd..4f6c1fc9c58d 100644 --- a/drivers/net/ethernet/google/gve/gve_main.c +++ b/drivers/net/ethernet/google/gve/gve_main.c @@ -514,14 +514,18 @@ static void gve_free_rings(struct gve_priv *priv) } } -int gve_alloc_page(struct device *dev, struct page **page, dma_addr_t *dma, +int gve_alloc_page(struct gve_priv *priv, struct device *dev, + struct page **page, dma_addr_t *dma, enum dma_data_direction dir) { *page = alloc_page(GFP_KERNEL); - if (!*page) + if (!*page) { + priv->page_alloc_fail++; return -ENOMEM; + } *dma = dma_map_page(dev, *page, 0, PAGE_SIZE, dir); if (dma_mapping_error(dev, *dma)) { + priv->dma_mapping_error++; put_page(*page); return -ENOMEM; } @@ -556,7 +560,7 @@ static int gve_alloc_queue_page_list(struct gve_priv *priv, u32 id, return -ENOMEM; for (i = 0; i < pages; i++) { - err = gve_alloc_page(&priv->pdev->dev, &qpl->pages[i], + err = gve_alloc_page(priv, &priv->pdev->dev, &qpl->pages[i], &qpl->page_buses[i], gve_qpl_dma_dir(priv, id)); /* caller handles clean up */ @@ -697,6 +701,7 @@ static int gve_open(struct net_device *dev) gve_turnup(priv); netif_carrier_on(dev); + priv->interface_up_cnt++; return 0; free_rings: @@ -738,6 +743,7 @@ static int gve_close(struct net_device *dev) gve_free_rings(priv); gve_free_qpls(priv); + priv->interface_down_cnt++; return 0; err: @@ -1047,6 +1053,9 @@ int gve_reset(struct gve_priv *priv, bool attempt_teardown) /* Set it all back up */ err = gve_reset_recovery(priv, was_up); gve_clear_reset_in_progress(priv); + priv->reset_cnt++; + priv->interface_up_cnt = 0; + priv->interface_down_cnt = 0; return err; } diff --git a/drivers/net/ethernet/google/gve/gve_rx.c b/drivers/net/ethernet/google/gve/gve_rx.c index 9f52e72ff641..008fa897a3e6 100644 --- a/drivers/net/ethernet/google/gve/gve_rx.c +++ b/drivers/net/ethernet/google/gve/gve_rx.c @@ -225,7 +225,8 @@ static enum pkt_hash_types gve_rss_type(__be16 pkt_flags) return PKT_HASH_TYPE_L2; } -static struct sk_buff *gve_rx_copy(struct net_device *dev, +static struct sk_buff *gve_rx_copy(struct gve_rx_ring *rx, + struct net_device *dev, struct napi_struct *napi, struct gve_rx_slot_page_info *page_info, u16 len) @@ -242,6 +243,11 @@ static struct sk_buff *gve_rx_copy(struct net_device *dev, skb_copy_to_linear_data(skb, va, len); skb->protocol = eth_type_trans(skb, dev); + + u64_stats_update_begin(&rx->statss); + rx->rx_copied_pkt++; + u64_stats_update_end(&rx->statss); + return skb; } @@ -284,8 +290,12 @@ static bool gve_rx(struct gve_rx_ring *rx, struct gve_rx_desc *rx_desc, u16 len; /* drop this packet */ - if (unlikely(rx_desc->flags_seq & GVE_RXF_ERR)) + if (unlikely(rx_desc->flags_seq & GVE_RXF_ERR)) { + u64_stats_update_begin(&rx->statss); + rx->rx_desc_err_dropped_pkt++; + u64_stats_update_end(&rx->statss); return true; + } len = be16_to_cpu(rx_desc->len) - GVE_RX_PAD; page_info = &rx->data.page_info[idx]; @@ -300,11 +310,14 @@ static bool gve_rx(struct gve_rx_ring *rx, struct gve_rx_desc *rx_desc, if (PAGE_SIZE == 4096) { if (len <= priv->rx_copybreak) { /* Just copy small packets */ - skb = gve_rx_copy(dev, napi, page_info, len); + skb = gve_rx_copy(rx, dev, napi, page_info, len); + u64_stats_update_begin(&rx->statss); + rx->rx_copybreak_pkt++; + u64_stats_update_end(&rx->statss); goto have_skb; } if (unlikely(!gve_can_recycle_pages(dev))) { - skb = gve_rx_copy(dev, napi, page_info, len); + skb = gve_rx_copy(rx, dev, napi, page_info, len); goto have_skb; } pagecount = page_count(page_info->page); @@ -314,8 +327,12 @@ static bool gve_rx(struct gve_rx_ring *rx, struct gve_rx_desc *rx_desc, * stack. */ skb = gve_rx_add_frags(dev, napi, page_info, len); - if (!skb) + if (!skb) { + u64_stats_update_begin(&rx->statss); + rx->rx_skb_alloc_fail++; + u64_stats_update_end(&rx->statss); return true; + } /* Make sure the kernel stack can't release the page */ get_page(page_info->page); /* "flip" to other packet buffer on this page */ @@ -324,21 +341,25 @@ static bool gve_rx(struct gve_rx_ring *rx, struct gve_rx_desc *rx_desc, /* We have previously passed the other half of this * page up the stack, but it has not yet been freed. */ - skb = gve_rx_copy(dev, napi, page_info, len); + skb = gve_rx_copy(rx, dev, napi, page_info, len); } else { WARN(pagecount < 1, "Pagecount should never be < 1"); return false; } } else { - skb = gve_rx_copy(dev, napi, page_info, len); + skb = gve_rx_copy(rx, dev, napi, page_info, len); } have_skb: /* We didn't manage to allocate an skb but we haven't had any * reset worthy failures. */ - if (!skb) + if (!skb) { + u64_stats_update_begin(&rx->statss); + rx->rx_skb_alloc_fail++; + u64_stats_update_end(&rx->statss); return true; + } if (likely(feat & NETIF_F_RXCSUM)) { /* NIC passes up the partial sum */ From patchwork Tue Sep 1 21:51:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Awogbemila X-Patchwork-Id: 1355421 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20161025 header.b=vktDh4NQ; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4Bh15112Cpz9sR4 for ; Wed, 2 Sep 2020 07:52:21 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729696AbgIAVwT (ORCPT ); Tue, 1 Sep 2020 17:52:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56772 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728288AbgIAVwC (ORCPT ); Tue, 1 Sep 2020 17:52:02 -0400 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [IPv6:2607:f8b0:4864:20::1049]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D4ED2C061247 for ; Tue, 1 Sep 2020 14:52:02 -0700 (PDT) Received: by mail-pj1-x1049.google.com with SMTP id v2so1246675pjh.3 for ; Tue, 01 Sep 2020 14:52:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=oHAl543n3+XdgnsNi5tJ8OOOXJ+WfyTUR/GGlQR3uTs=; b=vktDh4NQJh/GKaU/0D1XXYbKZHks1GVpGBef2hUa95MN23rV/TPNaqJELUVV9n5M/F gUyloCvdJm9WE8AwEra/bUB2KCZ8L3yOsTDO9qntjG4LoPd3eKJsdkg3j+sxHPOPTolO JcZmMuX2g7n29uwjDcg5PeCKYEl82tYZjBzPsCXXnbUFxwLTZucedGMxdKDYPn4d0IHC FOuoQRI3SYO7IxJ2eqX+La7FFexajfC9JEUVxoGWW7ccmaVBjBSrVOwAbGG1pS+V8Atn v/mxzINQIq4C/Idus7X0tt9E4LB24nEvYKcPCLNSxI40kxPMCVzSiMqkrrY4HbuuipTW uqyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=oHAl543n3+XdgnsNi5tJ8OOOXJ+WfyTUR/GGlQR3uTs=; b=gI+gensnzHR3HrW3Q5/Y1QObcUj7Vd166wBBe3+vyd5rDwSxYHkl3oJt3JzfP/ZcFS u9bopnYDwx0/oUnM/eutFMXCiE7gdUX2YYbMJ7PlsWxxklk65FgjIPi0E/hfWD1Lq0gf pHRlDd8ep6ZuAOHw3vKpB2sM+GMY+DoLBFxji1osyNs4UjWPdnl44IWwuRkOcF6sUuER d15d5NxMX3nYtrrsB5NRMYVGJKVS65GiIByO+uKqhVxncq8+aPInj30dJ9P6UIScsfa6 gudA8rZNp4h7H5hLfsxwdjcHkDiijdywOpscvy3MCrJ4BBnb35l+1gyNecd6skNXmbiZ TjDQ== X-Gm-Message-State: AOAM531t1gn1RpdgELYrh9JNi7mgvKqrx6ErMUDeCQXwf9zHWXepkpeE 3GW/iu9z6WHKF1wcR6XglW/Y4PMDHH1AkWQx7uH8apP0ISg/9l9eVOwifKO8MflnAOcevobrnzF pbU3kEYInCvT+yNKxQOIuBbreKyy5o5p2RQFHrcmrq9eRFdX/zej95CubIg/tLHEFZnX5AOKh X-Google-Smtp-Source: ABdhPJwDrKzl7/r8WfdMEfXo2hPMVkFhiY7AFCodQppaPUY3Vpi5HJI15dIX9srtlABQ307oFzKaspvACyIfVaaQ X-Received: from awogbemila.sea.corp.google.com ([2620:15c:100:202:1ea0:b8ff:fe73:6cc0]) (user=awogbemila job=sendgmr) by 2002:a17:90a:f117:: with SMTP id cc23mr3397642pjb.155.1598997122211; Tue, 01 Sep 2020 14:52:02 -0700 (PDT) Date: Tue, 1 Sep 2020 14:51:43 -0700 In-Reply-To: <20200901215149.2685117-1-awogbemila@google.com> Message-Id: <20200901215149.2685117-4-awogbemila@google.com> Mime-Version: 1.0 References: <20200901215149.2685117-1-awogbemila@google.com> X-Mailer: git-send-email 2.28.0.402.g5ffc5be6b7-goog Subject: [PATCH net-next v2 3/9] gve: Use dev_info/err instead of netif_info/err. From: David Awogbemila To: netdev@vger.kernel.org Cc: Catherine Sullivan , Yangchun Fu , David Awogbemila Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Catherine Sullivan Update the driver to use dev_info/err instead of netif_info/err. Reviewed-by: Yangchun Fu Signed-off-by: Catherine Sullivan Signed-off-by: David Awogbemila --- drivers/net/ethernet/google/gve/gve_adminq.c | 15 ++++++--------- drivers/net/ethernet/google/gve/gve_main.c | 14 +++++++++----- 2 files changed, 15 insertions(+), 14 deletions(-) diff --git a/drivers/net/ethernet/google/gve/gve_adminq.c b/drivers/net/ethernet/google/gve/gve_adminq.c index 529de756ff9b..d9aed217c1d6 100644 --- a/drivers/net/ethernet/google/gve/gve_adminq.c +++ b/drivers/net/ethernet/google/gve/gve_adminq.c @@ -334,8 +334,7 @@ int gve_adminq_describe_device(struct gve_priv *priv) priv->tx_desc_cnt = be16_to_cpu(descriptor->tx_queue_entries); if (priv->tx_desc_cnt * sizeof(priv->tx->desc[0]) < PAGE_SIZE) { - netif_err(priv, drv, priv->dev, "Tx desc count %d too low\n", - priv->tx_desc_cnt); + dev_err(&priv->pdev->dev, "Tx desc count %d too low\n", priv->tx_desc_cnt); err = -EINVAL; goto free_device_descriptor; } @@ -344,8 +343,7 @@ int gve_adminq_describe_device(struct gve_priv *priv) < PAGE_SIZE || priv->rx_desc_cnt * sizeof(priv->rx->data.data_ring[0]) < PAGE_SIZE) { - netif_err(priv, drv, priv->dev, "Rx desc count %d too low\n", - priv->rx_desc_cnt); + dev_err(&priv->pdev->dev, "Rx desc count %d too low\n", priv->rx_desc_cnt); err = -EINVAL; goto free_device_descriptor; } @@ -353,8 +351,7 @@ int gve_adminq_describe_device(struct gve_priv *priv) be64_to_cpu(descriptor->max_registered_pages); mtu = be16_to_cpu(descriptor->mtu); if (mtu < ETH_MIN_MTU) { - netif_err(priv, drv, priv->dev, "MTU %d below minimum MTU\n", - mtu); + dev_err(&priv->pdev->dev, "MTU %d below minimum MTU\n", mtu); err = -EINVAL; goto free_device_descriptor; } @@ -362,12 +359,12 @@ int gve_adminq_describe_device(struct gve_priv *priv) priv->num_event_counters = be16_to_cpu(descriptor->counters); ether_addr_copy(priv->dev->dev_addr, descriptor->mac); mac = descriptor->mac; - netif_info(priv, drv, priv->dev, "MAC addr: %pM\n", mac); + dev_info(&priv->pdev->dev, "MAC addr: %pM\n", mac); priv->tx_pages_per_qpl = be16_to_cpu(descriptor->tx_pages_per_qpl); priv->rx_pages_per_qpl = be16_to_cpu(descriptor->rx_pages_per_qpl); if (priv->rx_pages_per_qpl < priv->rx_desc_cnt) { - netif_err(priv, drv, priv->dev, "rx_pages_per_qpl cannot be smaller than rx_desc_cnt, setting rx_desc_cnt down to %d.\n", - priv->rx_pages_per_qpl); + dev_err(&priv->pdev->dev, "rx_pages_per_qpl cannot be smaller than rx_desc_cnt, setting rx_desc_cnt down to %d.\n", + priv->rx_pages_per_qpl); priv->rx_desc_cnt = priv->rx_pages_per_qpl; } priv->default_num_queues = be16_to_cpu(descriptor->default_num_queues); diff --git a/drivers/net/ethernet/google/gve/gve_main.c b/drivers/net/ethernet/google/gve/gve_main.c index 4f6c1fc9c58d..a0b8c1e8ed98 100644 --- a/drivers/net/ethernet/google/gve/gve_main.c +++ b/drivers/net/ethernet/google/gve/gve_main.c @@ -930,7 +930,7 @@ static int gve_init_priv(struct gve_priv *priv, bool skip_describe_device) priv->dev->max_mtu = PAGE_SIZE; err = gve_adminq_set_mtu(priv, priv->dev->mtu); if (err) { - netif_err(priv, drv, priv->dev, "Could not set mtu"); + dev_err(&priv->pdev->dev, "Could not set mtu"); goto err; } } @@ -970,10 +970,10 @@ static int gve_init_priv(struct gve_priv *priv, bool skip_describe_device) priv->rx_cfg.num_queues); } - netif_info(priv, drv, priv->dev, "TX queues %d, RX queues %d\n", - priv->tx_cfg.num_queues, priv->rx_cfg.num_queues); - netif_info(priv, drv, priv->dev, "Max TX queues %d, Max RX queues %d\n", - priv->tx_cfg.max_queues, priv->rx_cfg.max_queues); + dev_info(&priv->pdev->dev, "TX queues %d, RX queues %d\n", + priv->tx_cfg.num_queues, priv->rx_cfg.num_queues); + dev_info(&priv->pdev->dev, "Max TX queues %d, Max RX queues %d\n", + priv->tx_cfg.max_queues, priv->rx_cfg.max_queues); setup_device: err = gve_setup_device_resources(priv); @@ -1133,7 +1133,9 @@ static int gve_probe(struct pci_dev *pdev, const struct pci_device_id *ent) goto abort_with_db_bar; } SET_NETDEV_DEV(dev, &pdev->dev); + pci_set_drvdata(pdev, dev); + dev->ethtool_ops = &gve_ethtool_ops; dev->netdev_ops = &gve_netdev_ops; /* advertise features */ @@ -1160,6 +1162,7 @@ static int gve_probe(struct pci_dev *pdev, const struct pci_device_id *ent) priv->state_flags = 0x0; gve_set_probe_in_progress(priv); + priv->gve_wq = alloc_ordered_workqueue("gve", 0); if (!priv->gve_wq) { dev_err(&pdev->dev, "Could not allocate workqueue"); @@ -1181,6 +1184,7 @@ static int gve_probe(struct pci_dev *pdev, const struct pci_device_id *ent) dev_info(&pdev->dev, "GVE version %s\n", gve_version_str); gve_clear_probe_in_progress(priv); queue_work(priv->gve_wq, &priv->service_task); + return 0; abort_with_wq: From patchwork Tue Sep 1 21:51:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Awogbemila X-Patchwork-Id: 1355420 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20161025 header.b=qRBVToSZ; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4Bh1501CRTz9sR4 for ; Wed, 2 Sep 2020 07:52:20 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729673AbgIAVwR (ORCPT ); Tue, 1 Sep 2020 17:52:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56780 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728802AbgIAVwE (ORCPT ); Tue, 1 Sep 2020 17:52:04 -0400 Received: from mail-pj1-x104a.google.com (mail-pj1-x104a.google.com [IPv6:2607:f8b0:4864:20::104a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C1F16C061244 for ; Tue, 1 Sep 2020 14:52:04 -0700 (PDT) Received: by mail-pj1-x104a.google.com with SMTP id mu14so1132139pjb.7 for ; Tue, 01 Sep 2020 14:52:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=MPnRJ02GXcB1odHc4kcWJHJj72ELeZL12/LRGhlXO+U=; b=qRBVToSZfxNMEyfWWSIskImVuySB8g67Eb7zRAQTcCDaaIlUt5Zu4RbT/hBp1rqZvF 30MRkG2njqcJmm9ijRFjcsi8aXDYi3keFLMePQaQAtneawGSsJsigyW5D6F3XCUX1RW2 qD8siljKgA/ejJps++ABrv+a50r9vdLJWFGpkHtyll2aHoulqIHgvu6GXIJPAMmr9qY0 Lb9+lBd6mObbloeFLifYmP+XQ9UK85qc2RrZkfh/K3x404fSqcSSNzpj66tjggTUCtvI 1EMFTklY9u3yFK8i3xFbztJzAjrektSvQabr0sknQdtjDWL7a8OwxO0iNazDfHt7yeHP Ejyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=MPnRJ02GXcB1odHc4kcWJHJj72ELeZL12/LRGhlXO+U=; b=DXSbOGn9MW59FF3oDkvHwc5Xg2tgrkHs9tllu4BVWTutISiXfEXN49OpJ2ctBisesF xPtTbNaxahYUyibnWepAKqlxWS5P7Hkf0/Onx6pt9dxr9aKokU9UBqIzuKZ8ESUEPp0v rPd9/m+c8dMdNqipb3cPEqa1qCuf+yC/rpyVxge/csStbuZeyaV5bt8SlTkSy5CWTsPf nMe6CA4bqSazHvI0Le9JmaqU4D+1zpJwU87H7YwGTdczLubjVCvEQndvsTlpzDlkJbWy VbJYF0n7LEHm5Fa46Aoj/zLwVC9Ok7FyQht1GGBCPL47pjiq1ZemvCtRzBSQS42YiypS Vt2w== X-Gm-Message-State: AOAM530/MuNYtfZyi4kzVM2kjbkYrj5mTMx5+iWCDOTQyN6bVtAZieBc tg1BNNUIVPeKcegfAzKkhs9nngT1sRDAyp8FgCVu8UtB9QtgwNe6aBER+9k50Rlyuq6piyEXiYc ZWUwKdml71Yi7gvSIMuFuc+TJrEOkhMH5Iw6zJusBGh+SEohNz1SnJCdiEWQkkj/0S+ywGm0e X-Google-Smtp-Source: ABdhPJybTnBmQYOME4o5MQlGsHvt8TUROxTdAu5pz2MdMd1iFBW1+jqbaGsXALnGUdIaaBNNSQ8wBLA4himo/BT9 X-Received: from awogbemila.sea.corp.google.com ([2620:15c:100:202:1ea0:b8ff:fe73:6cc0]) (user=awogbemila job=sendgmr) by 2002:a62:7c43:0:b029:139:858b:8033 with SMTP id x64-20020a627c430000b0290139858b8033mr3845788pfc.3.1598997123848; Tue, 01 Sep 2020 14:52:03 -0700 (PDT) Date: Tue, 1 Sep 2020 14:51:44 -0700 In-Reply-To: <20200901215149.2685117-1-awogbemila@google.com> Message-Id: <20200901215149.2685117-5-awogbemila@google.com> Mime-Version: 1.0 References: <20200901215149.2685117-1-awogbemila@google.com> X-Mailer: git-send-email 2.28.0.402.g5ffc5be6b7-goog Subject: [PATCH net-next v2 4/9] gve: Add support for dma_mask register From: David Awogbemila To: netdev@vger.kernel.org Cc: Catherine Sullivan , Yangchun Fu , David Awogbemila Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Catherine Sullivan Add the dma_mask register and read it to set the dma_masks. gve_alloc_page will alloc_page with: GFP_DMA if priv->dma_mask is 24, GFP_DMA32 if priv->dma_mask is 32. Reviewed-by: Yangchun Fu Signed-off-by: Catherine Sullivan Signed-off-by: David Awogbemila --- drivers/net/ethernet/google/gve/gve.h | 6 ++- drivers/net/ethernet/google/gve/gve_main.c | 42 ++++++++++++------- .../net/ethernet/google/gve/gve_register.h | 3 +- 3 files changed, 34 insertions(+), 17 deletions(-) diff --git a/drivers/net/ethernet/google/gve/gve.h b/drivers/net/ethernet/google/gve/gve.h index 55b34b437764..37a3bbced36a 100644 --- a/drivers/net/ethernet/google/gve/gve.h +++ b/drivers/net/ethernet/google/gve/gve.h @@ -232,6 +232,9 @@ struct gve_priv { struct work_struct service_task; unsigned long service_task_flags; unsigned long state_flags; + + /* Gvnic device's dma mask, set during probe. */ + u8 dma_mask; }; enum gve_service_task_flags { @@ -451,8 +454,7 @@ static inline bool gve_can_recycle_pages(struct net_device *dev) } /* buffers */ -int gve_alloc_page(struct gve_priv *priv, struct device *dev, - struct page **page, dma_addr_t *dma, +int gve_alloc_page(struct gve_priv *priv, struct device *dev, struct page **page, dma_addr_t *dma, enum dma_data_direction); void gve_free_page(struct device *dev, struct page *page, dma_addr_t dma, enum dma_data_direction); diff --git a/drivers/net/ethernet/google/gve/gve_main.c b/drivers/net/ethernet/google/gve/gve_main.c index a0b8c1e8ed98..b176fcef19de 100644 --- a/drivers/net/ethernet/google/gve/gve_main.c +++ b/drivers/net/ethernet/google/gve/gve_main.c @@ -518,7 +518,14 @@ int gve_alloc_page(struct gve_priv *priv, struct device *dev, struct page **page, dma_addr_t *dma, enum dma_data_direction dir) { - *page = alloc_page(GFP_KERNEL); + gfp_t gfp_flags = GFP_KERNEL; + + if (priv->dma_mask == 24) + gfp_flags |= GFP_DMA; + else if (priv->dma_mask == 32) + gfp_flags |= GFP_DMA32; + + *page = alloc_page(gfp_flags); if (!*page) { priv->page_alloc_fail++; return -ENOMEM; @@ -1083,6 +1090,7 @@ static int gve_probe(struct pci_dev *pdev, const struct pci_device_id *ent) __be32 __iomem *db_bar; struct gve_registers __iomem *reg_bar; struct gve_priv *priv; + u8 dma_mask; int err; err = pci_enable_device(pdev); @@ -1095,19 +1103,6 @@ static int gve_probe(struct pci_dev *pdev, const struct pci_device_id *ent) pci_set_master(pdev); - err = pci_set_dma_mask(pdev, DMA_BIT_MASK(64)); - if (err) { - dev_err(&pdev->dev, "Failed to set dma mask: err=%d\n", err); - goto abort_with_pci_region; - } - - err = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64)); - if (err) { - dev_err(&pdev->dev, - "Failed to set consistent dma mask: err=%d\n", err); - goto abort_with_pci_region; - } - reg_bar = pci_iomap(pdev, GVE_REGISTER_BAR, 0); if (!reg_bar) { dev_err(&pdev->dev, "Failed to map pci bar!\n"); @@ -1122,10 +1117,28 @@ static int gve_probe(struct pci_dev *pdev, const struct pci_device_id *ent) goto abort_with_reg_bar; } + dma_mask = readb(®_bar->dma_mask); + // Default to 64 if the register isn't set + if (!dma_mask) + dma_mask = 64; gve_write_version(®_bar->driver_version); /* Get max queues to alloc etherdev */ max_rx_queues = ioread32be(®_bar->max_tx_queues); max_tx_queues = ioread32be(®_bar->max_rx_queues); + + err = pci_set_dma_mask(pdev, DMA_BIT_MASK(64)); + if (err) { + dev_err(&pdev->dev, "Failed to set dma mask: err=%d\n", err); + goto abort_with_reg_bar; + } + + err = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64)); + if (err) { + dev_err(&pdev->dev, + "Failed to set consistent dma mask: err=%d\n", err); + goto abort_with_reg_bar; + } + /* Alloc and setup the netdev and priv */ dev = alloc_etherdev_mqs(sizeof(*priv), max_tx_queues, max_rx_queues); if (!dev) { @@ -1160,6 +1173,7 @@ static int gve_probe(struct pci_dev *pdev, const struct pci_device_id *ent) priv->db_bar2 = db_bar; priv->service_task_flags = 0x0; priv->state_flags = 0x0; + priv->dma_mask = dma_mask; gve_set_probe_in_progress(priv); diff --git a/drivers/net/ethernet/google/gve/gve_register.h b/drivers/net/ethernet/google/gve/gve_register.h index 84ab8893aadd..fad8813d1bb1 100644 --- a/drivers/net/ethernet/google/gve/gve_register.h +++ b/drivers/net/ethernet/google/gve/gve_register.h @@ -16,7 +16,8 @@ struct gve_registers { __be32 adminq_pfn; __be32 adminq_doorbell; __be32 adminq_event_counter; - u8 reserved[3]; + u8 reserved[2]; + u8 dma_mask; u8 driver_version; }; From patchwork Tue Sep 1 21:51:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Awogbemila X-Patchwork-Id: 1355419 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20161025 header.b=Xw8lcmm7; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4Bh14v3hKKz9sR4 for ; Wed, 2 Sep 2020 07:52:15 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729600AbgIAVwO (ORCPT ); Tue, 1 Sep 2020 17:52:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56786 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729489AbgIAVwG (ORCPT ); Tue, 1 Sep 2020 17:52:06 -0400 Received: from mail-qv1-xf49.google.com (mail-qv1-xf49.google.com [IPv6:2607:f8b0:4864:20::f49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6E7C4C061244 for ; Tue, 1 Sep 2020 14:52:06 -0700 (PDT) Received: by mail-qv1-xf49.google.com with SMTP id y30so2018141qvy.9 for ; Tue, 01 Sep 2020 14:52:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=Co7q+eGb4eiMkacnhldQSHOllbo/vtMYCSiqbc9L+Rs=; b=Xw8lcmm7JuX9GemxOgo+x67YsR1yOBHv0Ugj0S8lS1ansD4l21xj7on93Jf/2GsYEj ZQuhAWuFOseUKV+C71tN7gozYyk6SNxs5d5C9q0nfzuC7olDWHp4BSkxhEFknqDS8TEJ p6CUrp+6WbbVpXgeSO5AGyzRT0la5nJWDVZWZIO6OMHlWMzKJFZ3RDtt3/rdJi2E/BkD h2spgGQ9Th3yhfjyzWSh1ekwFtOv+fSRIseWCiVMXVgBpqrCRp+ZfF26Tab7qaXhxxcq uQrxbwOSUHbfZx3Tvt6ezoMqYlnN5k1NVM4C+NBNYSW19cm45Sei7Prkq+LiJpuv0mNs OfAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Co7q+eGb4eiMkacnhldQSHOllbo/vtMYCSiqbc9L+Rs=; b=rI7KoyCHz4X5k0vvZ7o2RPyXs3yAbJFozAHI/S+LwYhGau8xSyR+28nL+LEQCCeqRM LGJXCsWRktLB9jJfEm6hxFranO2l83DktM7mJ/vCFca+v62WXmzcjPbNWaK7UjrxyJma Q6/NYzfIRCEXFaB4+gpGS0eH+kLJ/iprBzu8I/QXfntpZgNmjYPc+y+uDy++T5qh0g7O QKQEh9ve0q/wduOpxEYKIgIk9ZZaTDyX6qyHMn7a9TkjGue1w7oX5IKIAo/BgKmUILCf 3ATnqDhmsBpD/hUeOzabPrcYFir2yqNbWwCSW/I/wg3D4WJbDkI65LYKFz/7wZP/w/X2 hxKw== X-Gm-Message-State: AOAM533T/bEFiZykz1QdYfopaHXJutZWSeuZq4ga/0Em9kC7Y83Dn7GP H43f1kMEDLXHQrm7C2c6N6HUEiw1BvrfgHqXYZnd0e+OZBh26jllVefh51ApeEMHLiXBJwZe2FL GUcB+nrByVtSy2VU+3roHEmUOT804bHatJSi1onuWcQpe7a4xnEOc3FNglaZXs4C1W9RkWXRC X-Google-Smtp-Source: ABdhPJyzLxCcQx4aMRrPZRq4k/0y2JOuv9G4IC9mtae0ZlwTYobqadzDzLABuELRw6dV08PQbZNFOJXW4P+qjHrD X-Received: from awogbemila.sea.corp.google.com ([2620:15c:100:202:1ea0:b8ff:fe73:6cc0]) (user=awogbemila job=sendgmr) by 2002:a0c:eac5:: with SMTP id y5mr3386616qvp.2.1598997125406; Tue, 01 Sep 2020 14:52:05 -0700 (PDT) Date: Tue, 1 Sep 2020 14:51:45 -0700 In-Reply-To: <20200901215149.2685117-1-awogbemila@google.com> Message-Id: <20200901215149.2685117-6-awogbemila@google.com> Mime-Version: 1.0 References: <20200901215149.2685117-1-awogbemila@google.com> X-Mailer: git-send-email 2.28.0.402.g5ffc5be6b7-goog Subject: [PATCH net-next v2 5/9] gve: Add Gvnic stats AQ command and ethtool show/set-priv-flags. From: David Awogbemila To: netdev@vger.kernel.org Cc: Kuo Zhao , Yangchun Fu , David Awogbemila Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Kuo Zhao This adds functionality to report driver stats to Hypervisor. (Users may want to turn this feature off as a matter of privacy so a "report-stats" flag is addied as an ethtool priv option. It is also disabled by default.) A timer is also added so that when report-stats is enabled, stat are updated once every 20 seconds. Reviewed-by: Yangchun Fu Signed-off-by: Kuo Zhao Signed-off-by: David Awogbemila --- drivers/net/ethernet/google/gve/gve.h | 73 +++++++-- drivers/net/ethernet/google/gve/gve_adminq.c | 21 +++ drivers/net/ethernet/google/gve/gve_adminq.h | 38 +++++ drivers/net/ethernet/google/gve/gve_ethtool.c | 114 +++++++++++--- drivers/net/ethernet/google/gve/gve_main.c | 147 +++++++++++++++++- .../net/ethernet/google/gve/gve_register.h | 1 + 6 files changed, 362 insertions(+), 32 deletions(-) diff --git a/drivers/net/ethernet/google/gve/gve.h b/drivers/net/ethernet/google/gve/gve.h index 37a3bbced36a..9957a7d535ad 100644 --- a/drivers/net/ethernet/google/gve/gve.h +++ b/drivers/net/ethernet/google/gve/gve.h @@ -27,6 +27,13 @@ /* 1 for management, 1 for rx, 1 for tx */ #define GVE_MIN_MSIX 3 +/* Numbers of gve tx/rx stats in stats report. */ +#define GVE_TX_STATS_REPORT_NUM 5 +#define GVE_RX_STATS_REPORT_NUM 2 + +/* Interval to schedule a service task, 20000ms. */ +#define GVE_SERVICE_TIMER_PERIOD 20000 + /* Each slot in the desc ring has a 1:1 mapping to a slot in the data ring */ struct gve_rx_desc_queue { struct gve_rx_desc *desc_ring; /* the descriptor ring */ @@ -220,6 +227,7 @@ struct gve_priv { u32 adminq_destroy_rx_queue_cnt; u32 adminq_dcfg_device_resources_cnt; u32 adminq_set_driver_parameter_cnt; + u32 adminq_report_stats_cnt; /* Global stats */ u32 interface_up_cnt; /* count of times interface turned up since last reset */ @@ -227,27 +235,39 @@ struct gve_priv { u32 reset_cnt; /* count of reset */ u32 page_alloc_fail; /* count of page alloc fails */ u32 dma_mapping_error; /* count of dma mapping errors */ - struct workqueue_struct *gve_wq; struct work_struct service_task; unsigned long service_task_flags; unsigned long state_flags; + struct gve_stats_report *stats_report; + u64 stats_report_len; + dma_addr_t stats_report_bus; /* dma address for the stats report */ + unsigned long ethtool_flags; + + unsigned long service_timer_period; + struct timer_list service_timer; + /* Gvnic device's dma mask, set during probe. */ u8 dma_mask; }; -enum gve_service_task_flags { - GVE_PRIV_FLAGS_DO_RESET = BIT(1), - GVE_PRIV_FLAGS_RESET_IN_PROGRESS = BIT(2), - GVE_PRIV_FLAGS_PROBE_IN_PROGRESS = BIT(3), +enum gve_service_task_flags_bit { + GVE_PRIV_FLAGS_DO_RESET = 1, + GVE_PRIV_FLAGS_RESET_IN_PROGRESS = 2, + GVE_PRIV_FLAGS_PROBE_IN_PROGRESS = 3, + GVE_PRIV_FLAGS_DO_REPORT_STATS = 4, +}; + +enum gve_state_flags_bit { + GVE_PRIV_FLAGS_ADMIN_QUEUE_OK = 1, + GVE_PRIV_FLAGS_DEVICE_RESOURCES_OK = 2, + GVE_PRIV_FLAGS_DEVICE_RINGS_OK = 3, + GVE_PRIV_FLAGS_NAPI_ENABLED = 4, }; -enum gve_state_flags { - GVE_PRIV_FLAGS_ADMIN_QUEUE_OK = BIT(1), - GVE_PRIV_FLAGS_DEVICE_RESOURCES_OK = BIT(2), - GVE_PRIV_FLAGS_DEVICE_RINGS_OK = BIT(3), - GVE_PRIV_FLAGS_NAPI_ENABLED = BIT(4), +enum gve_ethtool_flags_bit { + GVE_PRIV_FLAGS_REPORT_STATS = 0, }; static inline bool gve_get_do_reset(struct gve_priv *priv) @@ -297,6 +317,22 @@ static inline void gve_clear_probe_in_progress(struct gve_priv *priv) clear_bit(GVE_PRIV_FLAGS_PROBE_IN_PROGRESS, &priv->service_task_flags); } +static inline bool gve_get_do_report_stats(struct gve_priv *priv) +{ + return test_bit(GVE_PRIV_FLAGS_DO_REPORT_STATS, + &priv->service_task_flags); +} + +static inline void gve_set_do_report_stats(struct gve_priv *priv) +{ + set_bit(GVE_PRIV_FLAGS_DO_REPORT_STATS, &priv->service_task_flags); +} + +static inline void gve_clear_do_report_stats(struct gve_priv *priv) +{ + clear_bit(GVE_PRIV_FLAGS_DO_REPORT_STATS, &priv->service_task_flags); +} + static inline bool gve_get_admin_queue_ok(struct gve_priv *priv) { return test_bit(GVE_PRIV_FLAGS_ADMIN_QUEUE_OK, &priv->state_flags); @@ -357,6 +393,21 @@ static inline void gve_clear_napi_enabled(struct gve_priv *priv) clear_bit(GVE_PRIV_FLAGS_NAPI_ENABLED, &priv->state_flags); } +static inline bool gve_get_report_stats(struct gve_priv *priv) +{ + return test_bit(GVE_PRIV_FLAGS_REPORT_STATS, &priv->ethtool_flags); +} + +static inline void gve_set_report_stats(struct gve_priv *priv) +{ + set_bit(GVE_PRIV_FLAGS_REPORT_STATS, &priv->ethtool_flags); +} + +static inline void gve_clear_report_stats(struct gve_priv *priv) +{ + clear_bit(GVE_PRIV_FLAGS_REPORT_STATS, &priv->ethtool_flags); +} + /* Returns the address of the ntfy_blocks irq doorbell */ static inline __be32 __iomem *gve_irq_doorbell(struct gve_priv *priv, @@ -478,6 +529,8 @@ int gve_reset(struct gve_priv *priv, bool attempt_teardown); int gve_adjust_queues(struct gve_priv *priv, struct gve_queue_config new_rx_config, struct gve_queue_config new_tx_config); +/* report stats handling */ +void gve_handle_report_stats(struct gve_priv *priv); /* exported by ethtool.c */ extern const struct ethtool_ops gve_ethtool_ops; /* needed by ethtool */ diff --git a/drivers/net/ethernet/google/gve/gve_adminq.c b/drivers/net/ethernet/google/gve/gve_adminq.c index d9aed217c1d6..69cdf92a2f21 100644 --- a/drivers/net/ethernet/google/gve/gve_adminq.c +++ b/drivers/net/ethernet/google/gve/gve_adminq.c @@ -35,6 +35,7 @@ int gve_adminq_alloc(struct device *dev, struct gve_priv *priv) priv->adminq_destroy_rx_queue_cnt = 0; priv->adminq_dcfg_device_resources_cnt = 0; priv->adminq_set_driver_parameter_cnt = 0; + priv->adminq_report_stats_cnt = 0; /* Setup Admin queue with the device */ iowrite32be(priv->adminq_bus_addr / PAGE_SIZE, @@ -183,6 +184,9 @@ int gve_adminq_execute_cmd(struct gve_priv *priv, case GVE_ADMINQ_SET_DRIVER_PARAMETER: priv->adminq_set_driver_parameter_cnt++; break; + case GVE_ADMINQ_REPORT_STATS: + priv->adminq_report_stats_cnt++; + break; default: dev_err(&priv->pdev->dev, "unknown AQ command opcode %d\n", opcode); } @@ -433,3 +437,20 @@ int gve_adminq_set_mtu(struct gve_priv *priv, u64 mtu) return gve_adminq_execute_cmd(priv, &cmd); } + +int gve_adminq_report_stats(struct gve_priv *priv, u64 stats_report_len, + dma_addr_t stats_report_addr, u64 interval) +{ + union gve_adminq_command cmd; + + memset(&cmd, 0, sizeof(cmd)); + cmd.opcode = cpu_to_be32(GVE_ADMINQ_REPORT_STATS); + cmd.report_stats = (struct gve_adminq_report_stats) { + .stats_report_len = cpu_to_be64(stats_report_len), + .stats_report_addr = cpu_to_be64(stats_report_addr), + .interval = cpu_to_be64(interval), + }; + + return gve_adminq_execute_cmd(priv, &cmd); +} + diff --git a/drivers/net/ethernet/google/gve/gve_adminq.h b/drivers/net/ethernet/google/gve/gve_adminq.h index 4dfa06edc0f8..b81a3bb76d5e 100644 --- a/drivers/net/ethernet/google/gve/gve_adminq.h +++ b/drivers/net/ethernet/google/gve/gve_adminq.h @@ -21,6 +21,7 @@ enum gve_adminq_opcodes { GVE_ADMINQ_DESTROY_RX_QUEUE = 0x8, GVE_ADMINQ_DECONFIGURE_DEVICE_RESOURCES = 0x9, GVE_ADMINQ_SET_DRIVER_PARAMETER = 0xB, + GVE_ADMINQ_REPORT_STATS = 0xC, }; /* Admin queue status codes */ @@ -172,6 +173,40 @@ struct gve_adminq_set_driver_parameter { static_assert(sizeof(struct gve_adminq_set_driver_parameter) == 16); +struct gve_adminq_report_stats { + __be64 stats_report_len; + __be64 stats_report_addr; + __be64 interval; +}; + +static_assert(sizeof(struct gve_adminq_report_stats) == 24); + +struct stats { + __be32 stat_name; + __be32 queue_id; + __be64 value; +}; + +static_assert(sizeof(struct stats) == 16); + +struct gve_stats_report { + __be64 written_count; + struct stats stats[0]; +}; + +static_assert(sizeof(struct gve_stats_report) == 8); + +enum gve_stat_names { + // stats from gve + TX_WAKE_CNT = 1, + TX_STOP_CNT = 2, + TX_FRAMES_SENT = 3, + TX_BYTES_SENT = 4, + TX_LAST_COMPLETION_PROCESSED = 5, + RX_NEXT_EXPECTED_SEQUENCE = 6, + RX_BUFFERS_POSTED = 7, +}; + union gve_adminq_command { struct { __be32 opcode; @@ -187,6 +222,7 @@ union gve_adminq_command { struct gve_adminq_register_page_list reg_page_list; struct gve_adminq_unregister_page_list unreg_page_list; struct gve_adminq_set_driver_parameter set_driver_param; + struct gve_adminq_report_stats report_stats; }; }; u8 reserved[64]; @@ -214,4 +250,6 @@ int gve_adminq_register_page_list(struct gve_priv *priv, struct gve_queue_page_list *qpl); int gve_adminq_unregister_page_list(struct gve_priv *priv, u32 page_list_id); int gve_adminq_set_mtu(struct gve_priv *priv, u64 mtu); +int gve_adminq_report_stats(struct gve_priv *priv, u64 stats_report_len, + dma_addr_t stats_report_addr, u64 interval); #endif /* _GVE_ADMINQ_H */ diff --git a/drivers/net/ethernet/google/gve/gve_ethtool.c b/drivers/net/ethernet/google/gve/gve_ethtool.c index 28d831d52701..8d70df2447ef 100644 --- a/drivers/net/ethernet/google/gve/gve_ethtool.c +++ b/drivers/net/ethernet/google/gve/gve_ethtool.c @@ -6,6 +6,7 @@ #include #include "gve.h" +#include "gve_adminq.h" static void gve_get_drvinfo(struct net_device *netdev, struct ethtool_drvinfo *info) @@ -56,12 +57,18 @@ static const char gve_gstrings_adminq_stats[][ETH_GSTRING_LEN] = { "adminq_create_tx_queue_cnt", "adminq_create_rx_queue_cnt", "adminq_destroy_tx_queue_cnt", "adminq_destroy_rx_queue_cnt", "adminq_dcfg_device_resources_cnt", "adminq_set_driver_parameter_cnt", + "adminq_report_stats_cnt", +}; + +static const char gve_gstrings_priv_flags[][ETH_GSTRING_LEN] = { + "report-stats", }; #define GVE_MAIN_STATS_LEN ARRAY_SIZE(gve_gstrings_main_stats) #define GVE_ADMINQ_STATS_LEN ARRAY_SIZE(gve_gstrings_adminq_stats) #define NUM_GVE_TX_CNTS ARRAY_SIZE(gve_gstrings_tx_stats) #define NUM_GVE_RX_CNTS ARRAY_SIZE(gve_gstrings_rx_stats) +#define GVE_PRIV_FLAGS_STR_LEN ARRAY_SIZE(gve_gstrings_priv_flags) static void gve_get_strings(struct net_device *netdev, u32 stringset, u8 *data) { @@ -69,30 +76,42 @@ static void gve_get_strings(struct net_device *netdev, u32 stringset, u8 *data) char *s = (char *)data; int i, j; - if (stringset != ETH_SS_STATS) - return; - - memcpy(s, *gve_gstrings_main_stats, - sizeof(gve_gstrings_main_stats)); - s += sizeof(gve_gstrings_main_stats); - - for (i = 0; i < priv->rx_cfg.num_queues; i++) { - for (j = 0; j < NUM_GVE_RX_CNTS; j++) { - snprintf(s, ETH_GSTRING_LEN, gve_gstrings_rx_stats[j], i); - s += ETH_GSTRING_LEN; + switch (stringset) { + case ETH_SS_STATS: + memcpy(s, *gve_gstrings_main_stats, + sizeof(gve_gstrings_main_stats)); + s += sizeof(gve_gstrings_main_stats); + + for (i = 0; i < priv->rx_cfg.num_queues; i++) { + for (j = 0; j < NUM_GVE_RX_CNTS; j++) { + snprintf(s, ETH_GSTRING_LEN, + gve_gstrings_rx_stats[j], i); + s += ETH_GSTRING_LEN; + } } - } - for (i = 0; i < priv->tx_cfg.num_queues; i++) { - for (j = 0; j < NUM_GVE_TX_CNTS; j++) { - snprintf(s, ETH_GSTRING_LEN, gve_gstrings_tx_stats[j], i); - s += ETH_GSTRING_LEN; + for (i = 0; i < priv->tx_cfg.num_queues; i++) { + for (j = 0; j < NUM_GVE_TX_CNTS; j++) { + snprintf(s, ETH_GSTRING_LEN, + gve_gstrings_tx_stats[j], i); + s += ETH_GSTRING_LEN; + } } - } - memcpy(s, *gve_gstrings_adminq_stats, - sizeof(gve_gstrings_adminq_stats)); - s += sizeof(gve_gstrings_adminq_stats); + memcpy(s, *gve_gstrings_adminq_stats, + sizeof(gve_gstrings_adminq_stats)); + s += sizeof(gve_gstrings_adminq_stats); + break; + + case ETH_SS_PRIV_FLAGS: + memcpy(s, *gve_gstrings_priv_flags, + sizeof(gve_gstrings_priv_flags)); + s += sizeof(gve_gstrings_priv_flags); + break; + + default: + break; + } } static int gve_get_sset_count(struct net_device *netdev, int sset) @@ -104,6 +123,8 @@ static int gve_get_sset_count(struct net_device *netdev, int sset) return GVE_MAIN_STATS_LEN + GVE_ADMINQ_STATS_LEN + (priv->rx_cfg.num_queues * NUM_GVE_RX_CNTS) + (priv->tx_cfg.num_queues * NUM_GVE_TX_CNTS); + case ETH_SS_PRIV_FLAGS: + return GVE_PRIV_FLAGS_STR_LEN; default: return -EOPNOTSUPP; } @@ -213,6 +234,7 @@ gve_get_ethtool_stats(struct net_device *netdev, } else { i += priv->rx_cfg.num_queues * NUM_GVE_RX_CNTS; } + /* walk TX rings */ if (priv->tx) { for (ring = 0; ring < priv->tx_cfg.num_queues; ring++) { @@ -235,6 +257,7 @@ gve_get_ethtool_stats(struct net_device *netdev, } else { i += priv->tx_cfg.num_queues * NUM_GVE_TX_CNTS; } + /* AQ Stats */ data[i++] = priv->adminq_prod_cnt; data[i++] = priv->adminq_cmd_fail; @@ -249,6 +272,7 @@ gve_get_ethtool_stats(struct net_device *netdev, data[i++] = priv->adminq_destroy_rx_queue_cnt; data[i++] = priv->adminq_dcfg_device_resources_cnt; data[i++] = priv->adminq_set_driver_parameter_cnt; + data[i++] = priv->adminq_report_stats_cnt; } static void gve_get_channels(struct net_device *netdev, @@ -353,6 +377,54 @@ static int gve_set_tunable(struct net_device *netdev, } } +static u32 gve_get_priv_flags(struct net_device *netdev) +{ + struct gve_priv *priv = netdev_priv(netdev); + u32 i, ret_flags = 0; + + for (i = 0; i < GVE_PRIV_FLAGS_STR_LEN; i++) { + if (priv->ethtool_flags & BIT(i)) + ret_flags |= BIT(i); + } + return ret_flags; +} + +static int gve_set_priv_flags(struct net_device *netdev, u32 flags) +{ + struct gve_priv *priv = netdev_priv(netdev); + u64 ori_flags, new_flags; + u32 i; + + ori_flags = READ_ONCE(priv->ethtool_flags); + new_flags = ori_flags; + + for (i = 0; i < GVE_PRIV_FLAGS_STR_LEN; i++) { + if (flags & BIT(i)) + new_flags |= BIT(i); + else + new_flags &= ~(BIT(i)); + priv->ethtool_flags = new_flags; + /* set report-stats */ + if (strcmp(gve_gstrings_priv_flags[i], "report-stats") == 0) { + /* update the stats when user turns report-stats on */ + if (flags & BIT(i)) + gve_handle_report_stats(priv); + /* zero off gve stats when report-stats turned off */ + if (!(flags & BIT(i)) && (ori_flags & BIT(i))) { + int tx_stats_num = GVE_TX_STATS_REPORT_NUM * + priv->tx_cfg.num_queues; + int rx_stats_num = GVE_RX_STATS_REPORT_NUM * + priv->rx_cfg.num_queues; + memset(priv->stats_report->stats, 0, + (tx_stats_num + rx_stats_num) * + sizeof(struct stats)); + } + } + } + + return 0; +} + const struct ethtool_ops gve_ethtool_ops = { .get_drvinfo = gve_get_drvinfo, .get_strings = gve_get_strings, @@ -367,4 +439,6 @@ const struct ethtool_ops gve_ethtool_ops = { .reset = gve_user_reset, .get_tunable = gve_get_tunable, .set_tunable = gve_set_tunable, + .get_priv_flags = gve_get_priv_flags, + .set_priv_flags = gve_set_priv_flags, }; diff --git a/drivers/net/ethernet/google/gve/gve_main.c b/drivers/net/ethernet/google/gve/gve_main.c index b176fcef19de..b76fc547cf5d 100644 --- a/drivers/net/ethernet/google/gve/gve_main.c +++ b/drivers/net/ethernet/google/gve/gve_main.c @@ -78,6 +78,59 @@ static void gve_free_counter_array(struct gve_priv *priv) priv->counter_array = NULL; } +static void gve_service_task_schedule(struct gve_priv *priv) +{ + if (!gve_get_probe_in_progress(priv) && + !gve_get_reset_in_progress(priv)) { + gve_set_do_report_stats(priv); + queue_work(priv->gve_wq, &priv->service_task); + } +} + +static void gve_service_timer(struct timer_list *t) +{ + struct gve_priv *priv = from_timer(priv, t, service_timer); + + mod_timer(&priv->service_timer, + round_jiffies(jiffies + + msecs_to_jiffies(priv->service_timer_period))); + gve_service_task_schedule(priv); +} + +static int gve_alloc_stats_report(struct gve_priv *priv) +{ + int tx_stats_num, rx_stats_num; + + tx_stats_num = (GVE_TX_STATS_REPORT_NUM) * + priv->tx_cfg.num_queues; + rx_stats_num = (GVE_RX_STATS_REPORT_NUM) * + priv->rx_cfg.num_queues; + priv->stats_report_len = sizeof(struct gve_stats_report) + + (tx_stats_num + rx_stats_num) * + sizeof(struct stats); + priv->stats_report = + dma_alloc_coherent(&priv->pdev->dev, priv->stats_report_len, + &priv->stats_report_bus, GFP_KERNEL); + if (!priv->stats_report) + return -ENOMEM; + /* Set up timer for periodic task */ + timer_setup(&priv->service_timer, gve_service_timer, 0); + priv->service_timer_period = GVE_SERVICE_TIMER_PERIOD; + /* Start the service task timer */ + mod_timer(&priv->service_timer, + round_jiffies(jiffies + + msecs_to_jiffies(priv->service_timer_period))); + return 0; +} + +static void gve_free_stats_report(struct gve_priv *priv) +{ + del_timer_sync(&priv->service_timer); + dma_free_coherent(&priv->pdev->dev, priv->stats_report_len, + priv->stats_report, priv->stats_report_bus); + priv->stats_report = NULL; +} + static irqreturn_t gve_mgmnt_intr(int irq, void *arg) { struct gve_priv *priv = arg; @@ -270,6 +323,9 @@ static int gve_setup_device_resources(struct gve_priv *priv) err = gve_alloc_notify_blocks(priv); if (err) goto abort_with_counter; + err = gve_alloc_stats_report(priv); + if (err) + goto abort_with_ntfy_blocks; err = gve_adminq_configure_device_resources(priv, priv->counter_array_bus, priv->num_event_counters, @@ -279,10 +335,18 @@ static int gve_setup_device_resources(struct gve_priv *priv) dev_err(&priv->pdev->dev, "could not setup device_resources: err=%d\n", err); err = -ENXIO; - goto abort_with_ntfy_blocks; + goto abort_with_stats_report; } + err = gve_adminq_report_stats(priv, priv->stats_report_len, + priv->stats_report_bus, + GVE_SERVICE_TIMER_PERIOD); + if (err) + dev_err(&priv->pdev->dev, + "Failed to report stats: err=%d\n", err); gve_set_device_resources_ok(priv); return 0; +abort_with_stats_report: + gve_free_stats_report(priv); abort_with_ntfy_blocks: gve_free_notify_blocks(priv); abort_with_counter: @@ -298,6 +362,13 @@ static void gve_teardown_device_resources(struct gve_priv *priv) /* Tell device its resources are being freed */ if (gve_get_device_resources_ok(priv)) { + /* detach the stats report */ + err = gve_adminq_report_stats(priv, 0, 0x0, GVE_SERVICE_TIMER_PERIOD); + if (err) { + dev_err(&priv->pdev->dev, + "Failed to detach stats report: err=%d\n", err); + gve_trigger_reset(priv); + } err = gve_adminq_deconfigure_device_resources(priv); if (err) { dev_err(&priv->pdev->dev, @@ -308,6 +379,7 @@ static void gve_teardown_device_resources(struct gve_priv *priv) } gve_free_counter_array(priv); gve_free_notify_blocks(priv); + gve_free_stats_report(priv); gve_clear_device_resources_ok(priv); } @@ -830,6 +902,7 @@ static void gve_turndown(struct gve_priv *priv) netif_tx_disable(priv->dev); gve_clear_napi_enabled(priv); + gve_clear_report_stats(priv); } static void gve_turnup(struct gve_priv *priv) @@ -880,6 +953,10 @@ static void gve_handle_status(struct gve_priv *priv, u32 status) dev_info(&priv->pdev->dev, "Device requested reset.\n"); gve_set_do_reset(priv); } + if (GVE_DEVICE_STATUS_REPORT_STATS_MASK & status) { + dev_info(&priv->pdev->dev, "Device report stats on.\n"); + gve_set_do_report_stats(priv); + } } static void gve_handle_reset(struct gve_priv *priv) @@ -898,7 +975,68 @@ static void gve_handle_reset(struct gve_priv *priv) } } -/* Handle NIC status register changes and reset requests */ +void gve_handle_report_stats(struct gve_priv *priv) +{ + int idx, stats_idx = 0, tx_bytes; + unsigned int start = 0; + struct stats *stats = priv->stats_report->stats; + + if (!gve_get_report_stats(priv)) + return; + + be64_add_cpu(&priv->stats_report->written_count, 1); + /* tx stats */ + if (priv->tx) { + for (idx = 0; idx < priv->tx_cfg.num_queues; idx++) { + do { + start = u64_stats_fetch_begin(&priv->tx[idx].statss); + tx_bytes = priv->tx[idx].bytes_done; + } while (u64_stats_fetch_retry(&priv->tx[idx].statss, start)); + stats[stats_idx++] = (struct stats) { + .stat_name = cpu_to_be32(TX_WAKE_CNT), + .value = cpu_to_be64(priv->tx[idx].wake_queue), + .queue_id = cpu_to_be32(idx), + }; + stats[stats_idx++] = (struct stats) { + .stat_name = cpu_to_be32(TX_STOP_CNT), + .value = cpu_to_be64(priv->tx[idx].stop_queue), + .queue_id = cpu_to_be32(idx), + }; + stats[stats_idx++] = (struct stats) { + .stat_name = cpu_to_be32(TX_FRAMES_SENT), + .value = cpu_to_be64(priv->tx[idx].req), + .queue_id = cpu_to_be32(idx), + }; + stats[stats_idx++] = (struct stats) { + .stat_name = cpu_to_be32(TX_BYTES_SENT), + .value = cpu_to_be64(tx_bytes), + .queue_id = cpu_to_be32(idx), + }; + stats[stats_idx++] = (struct stats) { + .stat_name = cpu_to_be32(TX_LAST_COMPLETION_PROCESSED), + .value = cpu_to_be64(priv->tx[idx].done), + .queue_id = cpu_to_be32(idx), + }; + } + } + /* rx stats */ + if (priv->rx) { + for (idx = 0; idx < priv->rx_cfg.num_queues; idx++) { + stats[stats_idx++] = (struct stats) { + .stat_name = cpu_to_be32(RX_NEXT_EXPECTED_SEQUENCE), + .value = cpu_to_be64(priv->rx[idx].desc.seqno), + .queue_id = cpu_to_be32(idx), + }; + stats[stats_idx++] = (struct stats) { + .stat_name = cpu_to_be32(RX_BUFFERS_POSTED), + .value = cpu_to_be64(priv->rx[0].fill_cnt), + .queue_id = cpu_to_be32(idx), + }; + } + } +} + +/* Handle NIC status register changes, reset requests and report stats */ static void gve_service_task(struct work_struct *work) { struct gve_priv *priv = container_of(work, struct gve_priv, @@ -908,6 +1046,10 @@ static void gve_service_task(struct work_struct *work) ioread32be(&priv->reg_bar0->device_status)); gve_handle_reset(priv); + if (gve_get_do_report_stats(priv)) { + gve_handle_report_stats(priv); + gve_clear_do_report_stats(priv); + } } static int gve_init_priv(struct gve_priv *priv, bool skip_describe_device) @@ -1173,6 +1315,7 @@ static int gve_probe(struct pci_dev *pdev, const struct pci_device_id *ent) priv->db_bar2 = db_bar; priv->service_task_flags = 0x0; priv->state_flags = 0x0; + priv->ethtool_flags = 0x0; priv->dma_mask = dma_mask; gve_set_probe_in_progress(priv); diff --git a/drivers/net/ethernet/google/gve/gve_register.h b/drivers/net/ethernet/google/gve/gve_register.h index fad8813d1bb1..776c2911842a 100644 --- a/drivers/net/ethernet/google/gve/gve_register.h +++ b/drivers/net/ethernet/google/gve/gve_register.h @@ -24,5 +24,6 @@ struct gve_registers { enum gve_device_status_flags { GVE_DEVICE_STATUS_RESET_MASK = BIT(1), GVE_DEVICE_STATUS_LINK_STATUS_MASK = BIT(2), + GVE_DEVICE_STATUS_REPORT_STATS_MASK = BIT(3), }; #endif /* _GVE_REGISTER_H_ */ From patchwork Tue Sep 1 21:51:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Awogbemila X-Patchwork-Id: 1355422 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20161025 header.b=lt2nq6eG; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4Bh1562sbVz9sTK for ; Wed, 2 Sep 2020 07:52:26 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729759AbgIAVwZ (ORCPT ); Tue, 1 Sep 2020 17:52:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56794 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727946AbgIAVwI (ORCPT ); Tue, 1 Sep 2020 17:52:08 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1C91CC061245 for ; Tue, 1 Sep 2020 14:52:08 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id j20so2704905ybt.10 for ; Tue, 01 Sep 2020 14:52:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=c20ktH6NmViBNAmU75r3cg0aTyG4UTJz4SQKC6m12jk=; b=lt2nq6eGWWdymykMHQ5DOmFQG5Q6XG3eSBSqqh/wavQqePoE48wAh8rtlP+bgRY6rK 6GfgB7CyKx+DApz3XdEFZtHZrS9Y/h2Ae4Rsk9djLLte+Q9sB/jG76Lh6mPfAGP1+4ks BJvHVKELX5Kam4X7yhpHMqr+3ZYmZSCEUB8cteCST2ooo1cRvtZYgIJ+hv6ApMhC1U9Y txT/DbnDlmOwPqdAZpPLd8tGmq0R7z+ByE1KaFkWtg3bnKTDAMBUa79ceVxdYyb+U/ML 8lv0/l0bvtfddZrI6ZHTqLUNrGgueewXjjLMGButB+Hg2QNRiu9BXTOfk7onfADmEGB9 VJQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=c20ktH6NmViBNAmU75r3cg0aTyG4UTJz4SQKC6m12jk=; b=LC9IAsaOT+2bn1j+f9Fnq4m+j87PZhj3/Hte0CJJ3EN2cNP3v4o0r9DACHTGkevqNg M3cry0NCA/C9flsdmFrH7U6WxIt7Cwdj3/Wg3xQaGT5EEuQiNNbbuZJ0q9Bj5ePGqWhL yi5nIKN1cg9hy/6b5umRkIHSk3x9pie04KlcJo0tuUo0BbHtd+pp5DOu5LsKa+UOAaIo lZFdscOMwYKjRnBF7gdh38YtMXX9Mb+78/FgJo8xe/1GfCRjUTSZFneSlbDSaPp6/+9u 3tcG7k53XnpRz8R88mB4DhpN24A7DMbLcsE0/KOTG9fhfU/ry68ih2kWz9j9LHuYWZPn MyiQ== X-Gm-Message-State: AOAM5327roSLpbGS/tiZ7zKblWTJ+kZ/WNqdNO3LViP4BejPcxLxGijF RgSy4BXexf+FTro0CuUmx0FGacMN+NvIeZEN3Qp6hf3fF7R4HqoA5PW0t58UPJEDfjFtBVFqOJQ 46L1pr89BeAtqqv1E9mZnagm/Yd91g2PL338Xdb4NyyRC10aWb+Aa5Hq9DDP14SyHGEay9QnD X-Google-Smtp-Source: ABdhPJz7dJDsRcYmnipGrb2aoIM3GSEb7unIJ711RdZSF46jnA3PQKFR4X2NK8AWo/p3JiFHbrL2XK0hG37dd+Ck X-Received: from awogbemila.sea.corp.google.com ([2620:15c:100:202:1ea0:b8ff:fe73:6cc0]) (user=awogbemila job=sendgmr) by 2002:a25:2e12:: with SMTP id u18mr5295029ybu.437.1598997127259; Tue, 01 Sep 2020 14:52:07 -0700 (PDT) Date: Tue, 1 Sep 2020 14:51:46 -0700 In-Reply-To: <20200901215149.2685117-1-awogbemila@google.com> Message-Id: <20200901215149.2685117-7-awogbemila@google.com> Mime-Version: 1.0 References: <20200901215149.2685117-1-awogbemila@google.com> X-Mailer: git-send-email 2.28.0.402.g5ffc5be6b7-goog Subject: [PATCH net-next v2 6/9] gve: NIC stats for report-stats and for ethtool From: David Awogbemila To: netdev@vger.kernel.org Cc: David Awogbemila Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org This adds per queue NIC stats to ethtool stats and to report-stats. These stats are always exposed to guest whether or not the report-stats flag is turned on. Signed-off-by: David Awogbemila --- drivers/net/ethernet/google/gve/gve.h | 4 + drivers/net/ethernet/google/gve/gve_adminq.h | 5 ++ drivers/net/ethernet/google/gve/gve_ethtool.c | 83 ++++++++++++++++++- drivers/net/ethernet/google/gve/gve_main.c | 4 +- 4 files changed, 93 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/google/gve/gve.h b/drivers/net/ethernet/google/gve/gve.h index 9957a7d535ad..bc54059f9b2e 100644 --- a/drivers/net/ethernet/google/gve/gve.h +++ b/drivers/net/ethernet/google/gve/gve.h @@ -31,6 +31,10 @@ #define GVE_TX_STATS_REPORT_NUM 5 #define GVE_RX_STATS_REPORT_NUM 2 +/* Numbers of NIC tx/rx stats in stats report. */ +#define NIC_TX_STATS_REPORT_NUM 0 +#define NIC_RX_STATS_REPORT_NUM 4 + /* Interval to schedule a service task, 20000ms. */ #define GVE_SERVICE_TIMER_PERIOD 20000 diff --git a/drivers/net/ethernet/google/gve/gve_adminq.h b/drivers/net/ethernet/google/gve/gve_adminq.h index b81a3bb76d5e..a6c8c29f0d13 100644 --- a/drivers/net/ethernet/google/gve/gve_adminq.h +++ b/drivers/net/ethernet/google/gve/gve_adminq.h @@ -205,6 +205,11 @@ enum gve_stat_names { TX_LAST_COMPLETION_PROCESSED = 5, RX_NEXT_EXPECTED_SEQUENCE = 6, RX_BUFFERS_POSTED = 7, + // stats from NIC + RX_QUEUE_DROP_CNT = 65, + RX_NO_BUFFERS_POSTED = 66, + RX_DROPS_PACKET_OVER_MRU = 67, + RX_DROPS_INVALID_CHECKSUM = 68, }; union gve_adminq_command { diff --git a/drivers/net/ethernet/google/gve/gve_ethtool.c b/drivers/net/ethernet/google/gve/gve_ethtool.c index 8d70df2447ef..db5642fe966f 100644 --- a/drivers/net/ethernet/google/gve/gve_ethtool.c +++ b/drivers/net/ethernet/google/gve/gve_ethtool.c @@ -43,6 +43,8 @@ static const char gve_gstrings_main_stats[][ETH_GSTRING_LEN] = { static const char gve_gstrings_rx_stats[][ETH_GSTRING_LEN] = { "rx_posted_desc[%u]", "rx_completed_desc[%u]", "rx_bytes[%u]", "rx_dropped_pkt[%u]", "rx_copybreak_pkt[%u]", "rx_copied_pkt[%u]", + "rx_queue_drop_cnt[%u]", "rx_no_buffers_posted[%u]", + "rx_drops_packet_over_mru[%u]", "rx_drops_invalid_checksum[%u]", }; static const char gve_gstrings_tx_stats[][ETH_GSTRING_LEN] = { @@ -138,14 +140,30 @@ gve_get_ethtool_stats(struct net_device *netdev, tmp_rx_desc_err_dropped_pkt, tmp_tx_pkts, tmp_tx_bytes; u64 rx_buf_alloc_fail, rx_desc_err_dropped_pkt, rx_pkts, rx_skb_alloc_fail, rx_bytes, tx_pkts, tx_bytes; + int stats_idx, base_stats_idx, max_stats_idx; + struct stats *report_stats; + int *rx_qid_to_stats_idx; + int *tx_qid_to_stats_idx; struct gve_priv *priv; + bool skip_nic_stats; unsigned int start; int ring; - int i; + int i, j; ASSERT_RTNL(); priv = netdev_priv(netdev); + report_stats = priv->stats_report->stats; + rx_qid_to_stats_idx = kmalloc_array(priv->rx_cfg.num_queues, + sizeof(int), GFP_KERNEL); + if (!rx_qid_to_stats_idx) + return; + tx_qid_to_stats_idx = kmalloc_array(priv->tx_cfg.num_queues, + sizeof(int), GFP_KERNEL); + if (!tx_qid_to_stats_idx) { + kfree(rx_qid_to_stats_idx); + return; + } for (rx_pkts = 0, rx_bytes = 0, rx_skb_alloc_fail = 0, rx_buf_alloc_fail = 0, rx_desc_err_dropped_pkt = 0, ring = 0; ring < priv->rx_cfg.num_queues; ring++) { @@ -206,6 +224,25 @@ gve_get_ethtool_stats(struct net_device *netdev, data[i++] = priv->dma_mapping_error; i = GVE_MAIN_STATS_LEN; + /* For rx cross-reporting stats, start from nic rx stats in report */ + base_stats_idx = GVE_TX_STATS_REPORT_NUM * priv->tx_cfg.num_queues + + GVE_RX_STATS_REPORT_NUM * priv->rx_cfg.num_queues; + max_stats_idx = NIC_RX_STATS_REPORT_NUM * priv->rx_cfg.num_queues + + base_stats_idx; + /* Preprocess the stats report for rx, map queue id to start index */ + skip_nic_stats = false; + for (stats_idx = base_stats_idx; stats_idx < max_stats_idx; + stats_idx += NIC_RX_STATS_REPORT_NUM) { + u32 stat_name = be32_to_cpu(report_stats[stats_idx].stat_name); + u32 queue_id = be32_to_cpu(report_stats[stats_idx].queue_id); + + if (stat_name == 0) { + /* no stats written by NIC yet */ + skip_nic_stats = true; + break; + } + rx_qid_to_stats_idx[queue_id] = stats_idx; + } /* walk RX rings */ if (priv->rx) { for (ring = 0; ring < priv->rx_cfg.num_queues; ring++) { @@ -230,11 +267,41 @@ gve_get_ethtool_stats(struct net_device *netdev, tmp_rx_desc_err_dropped_pkt; data[i++] = rx->rx_copybreak_pkt; data[i++] = rx->rx_copied_pkt; + /* stats from NIC */ + if (skip_nic_stats) { + /* skip NIC rx stats */ + i += NIC_RX_STATS_REPORT_NUM; + continue; + } + for (j = 0; j < NIC_RX_STATS_REPORT_NUM; j++) { + u64 value = + be64_to_cpu(report_stats[rx_qid_to_stats_idx[ring] + j].value); + + data[i++] = value; + } } } else { i += priv->rx_cfg.num_queues * NUM_GVE_RX_CNTS; } + /* For tx cross-reporting stats, start from nic tx stats in report */ + base_stats_idx = max_stats_idx; + max_stats_idx = NIC_TX_STATS_REPORT_NUM * priv->tx_cfg.num_queues + + max_stats_idx; + /* Preprocess the stats report for tx, map queue id to start index */ + skip_nic_stats = false; + for (stats_idx = base_stats_idx; stats_idx < max_stats_idx; + stats_idx += NIC_TX_STATS_REPORT_NUM) { + u32 stat_name = be32_to_cpu(report_stats[stats_idx].stat_name); + u32 queue_id = be32_to_cpu(report_stats[stats_idx].queue_id); + + if (stat_name == 0) { + /* no stats written by NIC yet */ + skip_nic_stats = true; + break; + } + tx_qid_to_stats_idx[queue_id] = stats_idx; + } /* walk TX rings */ if (priv->tx) { for (ring = 0; ring < priv->tx_cfg.num_queues; ring++) { @@ -253,11 +320,25 @@ gve_get_ethtool_stats(struct net_device *netdev, data[i++] = tx->stop_queue; data[i++] = be32_to_cpu(gve_tx_load_event_counter(priv, tx)); + /* stats from NIC */ + if (skip_nic_stats) { + /* skip NIC tx stats */ + i += NIC_TX_STATS_REPORT_NUM; + continue; + } + for (j = 0; j < NIC_TX_STATS_REPORT_NUM; j++) { + u64 value = + be64_to_cpu(report_stats[tx_qid_to_stats_idx[ring] + j].value); + data[i++] = value; + } } } else { i += priv->tx_cfg.num_queues * NUM_GVE_TX_CNTS; } + kfree(rx_qid_to_stats_idx); + kfree(tx_qid_to_stats_idx); + /* AQ Stats */ data[i++] = priv->adminq_prod_cnt; data[i++] = priv->adminq_cmd_fail; diff --git a/drivers/net/ethernet/google/gve/gve_main.c b/drivers/net/ethernet/google/gve/gve_main.c index b76fc547cf5d..0c94dda67ba4 100644 --- a/drivers/net/ethernet/google/gve/gve_main.c +++ b/drivers/net/ethernet/google/gve/gve_main.c @@ -101,9 +101,9 @@ static int gve_alloc_stats_report(struct gve_priv *priv) { int tx_stats_num, rx_stats_num; - tx_stats_num = (GVE_TX_STATS_REPORT_NUM) * + tx_stats_num = (GVE_TX_STATS_REPORT_NUM + NIC_TX_STATS_REPORT_NUM) * priv->tx_cfg.num_queues; - rx_stats_num = (GVE_RX_STATS_REPORT_NUM) * + rx_stats_num = (GVE_RX_STATS_REPORT_NUM + NIC_RX_STATS_REPORT_NUM) * priv->rx_cfg.num_queues; priv->stats_report_len = sizeof(struct gve_stats_report) + (tx_stats_num + rx_stats_num) * From patchwork Tue Sep 1 21:51:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Awogbemila X-Patchwork-Id: 1355423 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20161025 header.b=d79ZzF1v; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4Bh1566HVGz9sTM for ; Wed, 2 Sep 2020 07:52:26 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729819AbgIAVw0 (ORCPT ); Tue, 1 Sep 2020 17:52:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56796 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728677AbgIAVwK (ORCPT ); Tue, 1 Sep 2020 17:52:10 -0400 Received: from mail-pf1-x449.google.com (mail-pf1-x449.google.com [IPv6:2607:f8b0:4864:20::449]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8F987C061246 for ; Tue, 1 Sep 2020 14:52:09 -0700 (PDT) Received: by mail-pf1-x449.google.com with SMTP id y4so1480346pff.13 for ; Tue, 01 Sep 2020 14:52:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=MQQluZ3SVhkcO/WV0C9DPGDlzAdlXQlLzeaTji7HUak=; b=d79ZzF1vtJwqYf5xZWR5AzSRj8lUjwmbQeh904YVH21w/MbNOZavsWH9moZKRtOyKJ bBL9ytmUyHxC24tp7ZrVK1FHdJz3L2xPYZG6L5uDN91ATKQ7VT1qBBg0Sb1YxqIbdWt4 9fYMstWYpkZMdNzPQPjbYrNceNWhXOuq5P2rJDWuHkflICwDcp+MI/SlWX4Y09V8kjaH S6Uc5Rz3M+AWwbWIkku7eDI9clNEJ9XVW+Pus1lGzbY3NVpitTupTuUDofs1qdQQJOOE bs6HjMQV4dxa48kNeAa4FyIM3F87fF4ohNFdCnPBtAwxkCIF/3fEVydhRgcjjzzAjQqz IbmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=MQQluZ3SVhkcO/WV0C9DPGDlzAdlXQlLzeaTji7HUak=; b=tEFOdgBrrlxAcggobTGEwrgW3GoFaPBmtnIy/wLo21poa8laX9GtTePmQ23+clS5M+ GOJ+Mam8IRrKXIVxtisIOZWDYRKexuMZuUTLtHzIfkDse3K5mpkAxn2wzbaJRKLWfEi3 Of+K9g9/g/JIZ2z1fEWenzdSgLFuJ5azce+JwDE5ZWcew1RGMYeJgfKs2UwbNSALSQxz SNu04MUS25Yx9pw0pUP81MKeLZgazTYkqpaQxzxohCkoadmTHRGP1y5nGVkAKOszoE7l tDCMhgRexvkQigRm6jFzUWeSUhHA4zhfw+ieeTIEnCm2gn+VzYojNk+aKg8nmbe50gvx vaMA== X-Gm-Message-State: AOAM532JnhUbWgTXX6srapovnYO2+seFxC+q8dkeSR/aYtz5KjakLMDP 6YTuzgzHBjaRF5ABdopC6khwUMT7k4V1C52Tt0BU9HR0aYOLdNug4dlJ0RqnUyKdUxZoud6rxS9 eFDAW+D0Gj8qIlUi69acaYuoLKHyfoh8i8bxcTsjlyoOUTAOpLGGH7s0ErWPZK1aN0veAAbuE X-Google-Smtp-Source: ABdhPJyiWcFEMxpT0+4pipdqA0+xSggAtzrRMPtHuQ6nuGXCk2Iy6Yo5h3dS84opcC0g+/0/gz39mdy6MyXG0Spe X-Received: from awogbemila.sea.corp.google.com ([2620:15c:100:202:1ea0:b8ff:fe73:6cc0]) (user=awogbemila job=sendgmr) by 2002:a62:7e4e:0:b029:137:3e8f:c31c with SMTP id z75-20020a627e4e0000b02901373e8fc31cmr3762513pfc.0.1598997129005; Tue, 01 Sep 2020 14:52:09 -0700 (PDT) Date: Tue, 1 Sep 2020 14:51:47 -0700 In-Reply-To: <20200901215149.2685117-1-awogbemila@google.com> Message-Id: <20200901215149.2685117-8-awogbemila@google.com> Mime-Version: 1.0 References: <20200901215149.2685117-1-awogbemila@google.com> X-Mailer: git-send-email 2.28.0.402.g5ffc5be6b7-goog Subject: [PATCH net-next v2 7/9] gve: Batch AQ commands for creating and destroying queues. From: David Awogbemila To: netdev@vger.kernel.org Cc: Sagi Shahar , Yangchun Fu , David Awogbemila Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Sagi Shahar Adds support for batching AQ commands and uses it for creating and destroying queues. Reviewed-by: Yangchun Fu Signed-off-by: Sagi Shahar Signed-off-by: David Awogbemila --- drivers/net/ethernet/google/gve/gve_adminq.c | 188 ++++++++++++++++--- drivers/net/ethernet/google/gve/gve_adminq.h | 10 +- drivers/net/ethernet/google/gve/gve_main.c | 94 +++++----- 3 files changed, 211 insertions(+), 81 deletions(-) diff --git a/drivers/net/ethernet/google/gve/gve_adminq.c b/drivers/net/ethernet/google/gve/gve_adminq.c index 69cdf92a2f21..341a17b36f06 100644 --- a/drivers/net/ethernet/google/gve/gve_adminq.c +++ b/drivers/net/ethernet/google/gve/gve_adminq.c @@ -135,20 +135,71 @@ static int gve_adminq_parse_err(struct gve_priv *priv, u32 status) } } +/* Flushes all AQ commands currently queued and waits for them to complete. + * If there are failures, it will return the first error. + */ +static int gve_adminq_kick_and_wait(struct gve_priv *priv) +{ + u32 tail, head; + int i; + + tail = ioread32be(&priv->reg_bar0->adminq_event_counter); + head = priv->adminq_prod_cnt; + + gve_adminq_kick_cmd(priv, head); + if (!gve_adminq_wait_for_cmd(priv, head)) { + dev_err(&priv->pdev->dev, "AQ commands timed out, need to reset AQ\n"); + priv->adminq_timeouts++; + return -ENOTRECOVERABLE; + } + + for (i = tail; i < head; i++) { + union gve_adminq_command *cmd; + u32 status, err; + + cmd = &priv->adminq[i & priv->adminq_mask]; + status = be32_to_cpu(READ_ONCE(cmd->status)); + err = gve_adminq_parse_err(priv, status); + if (err) + // Return the first error if we failed. + return err; + } + + return 0; +} + /* This function is not threadsafe - the caller is responsible for any * necessary locks. */ -int gve_adminq_execute_cmd(struct gve_priv *priv, - union gve_adminq_command *cmd_orig) +static int gve_adminq_issue_cmd(struct gve_priv *priv, + union gve_adminq_command *cmd_orig) { union gve_adminq_command *cmd; - u32 status = 0; - u32 prod_cnt; u32 opcode; + u32 tail; + + tail = ioread32be(&priv->reg_bar0->adminq_event_counter); + + // Check if next command will overflow the buffer. + if (((priv->adminq_prod_cnt + 1) & priv->adminq_mask) == tail) { + int err; + + // Flush existing commands to make room. + err = gve_adminq_kick_and_wait(priv); + if (err) + return err; + + // Retry. + tail = ioread32be(&priv->reg_bar0->adminq_event_counter); + if (((priv->adminq_prod_cnt + 1) & priv->adminq_mask) == tail) { + // This should never happen. We just flushed the + // command queue so there should be enough space. + return -ENOMEM; + } + } cmd = &priv->adminq[priv->adminq_prod_cnt & priv->adminq_mask]; priv->adminq_prod_cnt++; - prod_cnt = priv->adminq_prod_cnt; memcpy(cmd, cmd_orig, sizeof(*cmd_orig)); opcode = be32_to_cpu(READ_ONCE(cmd->opcode)); @@ -191,16 +242,30 @@ int gve_adminq_execute_cmd(struct gve_priv *priv, dev_err(&priv->pdev->dev, "unknown AQ command opcode %d\n", opcode); } - gve_adminq_kick_cmd(priv, prod_cnt); - if (!gve_adminq_wait_for_cmd(priv, prod_cnt)) { - dev_err(&priv->pdev->dev, "AQ command timed out, need to reset AQ\n"); - priv->adminq_timeouts++; - return -ENOTRECOVERABLE; - } + return 0; +} - memcpy(cmd_orig, cmd, sizeof(*cmd)); - status = be32_to_cpu(READ_ONCE(cmd->status)); - return gve_adminq_parse_err(priv, status); +/* This function is not threadsafe - the caller is responsible for any + * necessary locks. + * The caller is also responsible for making sure there are no commands + * waiting to be executed. + */ +static int gve_adminq_execute_cmd(struct gve_priv *priv, union gve_adminq_command *cmd_orig) +{ + u32 tail, head; + int err; + + tail = ioread32be(&priv->reg_bar0->adminq_event_counter); + head = priv->adminq_prod_cnt; + if (tail != head) + // This is not a valid path + return -EINVAL; + + err = gve_adminq_issue_cmd(priv, cmd_orig); + if (err) + return err; + + return gve_adminq_kick_and_wait(priv); } /* The device specifies that the management vector can either be the first irq @@ -245,29 +310,50 @@ int gve_adminq_deconfigure_device_resources(struct gve_priv *priv) return gve_adminq_execute_cmd(priv, &cmd); } -int gve_adminq_create_tx_queue(struct gve_priv *priv, u32 queue_index) +static int gve_adminq_create_tx_queue(struct gve_priv *priv, u32 queue_index) { struct gve_tx_ring *tx = &priv->tx[queue_index]; union gve_adminq_command cmd; + int err; memset(&cmd, 0, sizeof(cmd)); cmd.opcode = cpu_to_be32(GVE_ADMINQ_CREATE_TX_QUEUE); cmd.create_tx_queue = (struct gve_adminq_create_tx_queue) { .queue_id = cpu_to_be32(queue_index), .reserved = 0, - .queue_resources_addr = cpu_to_be64(tx->q_resources_bus), + .queue_resources_addr = + cpu_to_be64(tx->q_resources_bus), .tx_ring_addr = cpu_to_be64(tx->bus), .queue_page_list_id = cpu_to_be32(tx->tx_fifo.qpl->id), .ntfy_id = cpu_to_be32(tx->ntfy_id), }; - return gve_adminq_execute_cmd(priv, &cmd); + err = gve_adminq_issue_cmd(priv, &cmd); + if (err) + return err; + + return 0; } -int gve_adminq_create_rx_queue(struct gve_priv *priv, u32 queue_index) +int gve_adminq_create_tx_queues(struct gve_priv *priv, u32 num_queues) +{ + int err; + int i; + + for (i = 0; i < num_queues; i++) { + err = gve_adminq_create_tx_queue(priv, i); + if (err) + return err; + } + + return gve_adminq_kick_and_wait(priv); +} + +static int gve_adminq_create_rx_queue(struct gve_priv *priv, u32 queue_index) { struct gve_rx_ring *rx = &priv->rx[queue_index]; union gve_adminq_command cmd; + int err; memset(&cmd, 0, sizeof(cmd)); cmd.opcode = cpu_to_be32(GVE_ADMINQ_CREATE_RX_QUEUE); @@ -282,12 +368,31 @@ int gve_adminq_create_rx_queue(struct gve_priv *priv, u32 queue_index) .queue_page_list_id = cpu_to_be32(rx->data.qpl->id), }; - return gve_adminq_execute_cmd(priv, &cmd); + err = gve_adminq_issue_cmd(priv, &cmd); + if (err) + return err; + + return 0; } -int gve_adminq_destroy_tx_queue(struct gve_priv *priv, u32 queue_index) +int gve_adminq_create_rx_queues(struct gve_priv *priv, u32 num_queues) +{ + int err; + int i; + + for (i = 0; i < num_queues; i++) { + err = gve_adminq_create_rx_queue(priv, i); + if (err) + return err; + } + + return gve_adminq_kick_and_wait(priv); +} + +static int gve_adminq_destroy_tx_queue(struct gve_priv *priv, u32 queue_index) { union gve_adminq_command cmd; + int err; memset(&cmd, 0, sizeof(cmd)); cmd.opcode = cpu_to_be32(GVE_ADMINQ_DESTROY_TX_QUEUE); @@ -295,12 +400,31 @@ int gve_adminq_destroy_tx_queue(struct gve_priv *priv, u32 queue_index) .queue_id = cpu_to_be32(queue_index), }; - return gve_adminq_execute_cmd(priv, &cmd); + err = gve_adminq_issue_cmd(priv, &cmd); + if (err) + return err; + + return 0; } -int gve_adminq_destroy_rx_queue(struct gve_priv *priv, u32 queue_index) +int gve_adminq_destroy_tx_queues(struct gve_priv *priv, u32 num_queues) +{ + int err; + int i; + + for (i = 0; i < num_queues; i++) { + err = gve_adminq_destroy_tx_queue(priv, i); + if (err) + return err; + } + + return gve_adminq_kick_and_wait(priv); +} + +static int gve_adminq_destroy_rx_queue(struct gve_priv *priv, u32 queue_index) { union gve_adminq_command cmd; + int err; memset(&cmd, 0, sizeof(cmd)); cmd.opcode = cpu_to_be32(GVE_ADMINQ_DESTROY_RX_QUEUE); @@ -308,7 +432,25 @@ int gve_adminq_destroy_rx_queue(struct gve_priv *priv, u32 queue_index) .queue_id = cpu_to_be32(queue_index), }; - return gve_adminq_execute_cmd(priv, &cmd); + err = gve_adminq_issue_cmd(priv, &cmd); + if (err) + return err; + + return 0; +} + +int gve_adminq_destroy_rx_queues(struct gve_priv *priv, u32 num_queues) +{ + int err; + int i; + + for (i = 0; i < num_queues; i++) { + err = gve_adminq_destroy_rx_queue(priv, i); + if (err) + return err; + } + + return gve_adminq_kick_and_wait(priv); } int gve_adminq_describe_device(struct gve_priv *priv) diff --git a/drivers/net/ethernet/google/gve/gve_adminq.h b/drivers/net/ethernet/google/gve/gve_adminq.h index a6c8c29f0d13..784830f75b7c 100644 --- a/drivers/net/ethernet/google/gve/gve_adminq.h +++ b/drivers/net/ethernet/google/gve/gve_adminq.h @@ -238,8 +238,6 @@ static_assert(sizeof(union gve_adminq_command) == 64); int gve_adminq_alloc(struct device *dev, struct gve_priv *priv); void gve_adminq_free(struct device *dev, struct gve_priv *priv); void gve_adminq_release(struct gve_priv *priv); -int gve_adminq_execute_cmd(struct gve_priv *priv, - union gve_adminq_command *cmd_orig); int gve_adminq_describe_device(struct gve_priv *priv); int gve_adminq_configure_device_resources(struct gve_priv *priv, dma_addr_t counter_array_bus_addr, @@ -247,10 +245,10 @@ int gve_adminq_configure_device_resources(struct gve_priv *priv, dma_addr_t db_array_bus_addr, u32 num_ntfy_blks); int gve_adminq_deconfigure_device_resources(struct gve_priv *priv); -int gve_adminq_create_tx_queue(struct gve_priv *priv, u32 queue_id); -int gve_adminq_destroy_tx_queue(struct gve_priv *priv, u32 queue_id); -int gve_adminq_create_rx_queue(struct gve_priv *priv, u32 queue_id); -int gve_adminq_destroy_rx_queue(struct gve_priv *priv, u32 queue_id); +int gve_adminq_create_tx_queues(struct gve_priv *priv, u32 num_queues); +int gve_adminq_destroy_tx_queues(struct gve_priv *priv, u32 queue_id); +int gve_adminq_create_rx_queues(struct gve_priv *priv, u32 num_queues); +int gve_adminq_destroy_rx_queues(struct gve_priv *priv, u32 queue_id); int gve_adminq_register_page_list(struct gve_priv *priv, struct gve_queue_page_list *qpl); int gve_adminq_unregister_page_list(struct gve_priv *priv, u32 page_list_id); diff --git a/drivers/net/ethernet/google/gve/gve_main.c b/drivers/net/ethernet/google/gve/gve_main.c index 0c94dda67ba4..d5019132874d 100644 --- a/drivers/net/ethernet/google/gve/gve_main.c +++ b/drivers/net/ethernet/google/gve/gve_main.c @@ -443,36 +443,37 @@ static int gve_create_rings(struct gve_priv *priv) int err; int i; - for (i = 0; i < priv->tx_cfg.num_queues; i++) { - err = gve_adminq_create_tx_queue(priv, i); - if (err) { - netif_err(priv, drv, priv->dev, "failed to create tx queue %d\n", - i); - /* This failure will trigger a reset - no need to clean - * up - */ - return err; - } - netif_dbg(priv, drv, priv->dev, "created tx queue %d\n", i); + err = gve_adminq_create_tx_queues(priv, priv->tx_cfg.num_queues); + if (err) { + netif_err(priv, drv, priv->dev, "failed to create %d tx queues\n", + priv->tx_cfg.num_queues); + /* This failure will trigger a reset - no need to clean + * up + */ + return err; } - for (i = 0; i < priv->rx_cfg.num_queues; i++) { - err = gve_adminq_create_rx_queue(priv, i); - if (err) { - netif_err(priv, drv, priv->dev, "failed to create rx queue %d\n", - i); - /* This failure will trigger a reset - no need to clean - * up - */ - return err; - } - /* Rx data ring has been prefilled with packet buffers at - * queue allocation time. - * Write the doorbell to provide descriptor slots and packet - * buffers to the NIC. + netif_dbg(priv, drv, priv->dev, "created %d tx queues\n", + priv->tx_cfg.num_queues); + + err = gve_adminq_create_rx_queues(priv, priv->rx_cfg.num_queues); + if (err) { + netif_err(priv, drv, priv->dev, "failed to create %d rx queues\n", + priv->rx_cfg.num_queues); + /* This failure will trigger a reset - no need to clean + * up */ - gve_rx_write_doorbell(priv, &priv->rx[i]); - netif_dbg(priv, drv, priv->dev, "created rx queue %d\n", i); + return err; } + netif_dbg(priv, drv, priv->dev, "created %d rx queues\n", + priv->rx_cfg.num_queues); + + /* Rx data ring has been prefilled with packet buffers at queue + * allocation time. + * Write the doorbell to provide descriptor slots and packet buffers + * to the NIC. + */ + for (i = 0; i < priv->rx_cfg.num_queues; i++) + gve_rx_write_doorbell(priv, &priv->rx[i]); return 0; } @@ -530,34 +531,23 @@ static int gve_alloc_rings(struct gve_priv *priv) static int gve_destroy_rings(struct gve_priv *priv) { int err; - int i; - for (i = 0; i < priv->tx_cfg.num_queues; i++) { - err = gve_adminq_destroy_tx_queue(priv, i); - if (err) { - netif_err(priv, drv, priv->dev, - "failed to destroy tx queue %d\n", - i); - /* This failure will trigger a reset - no need to clean - * up - */ - return err; - } - netif_dbg(priv, drv, priv->dev, "destroyed tx queue %d\n", i); + err = gve_adminq_destroy_tx_queues(priv, priv->tx_cfg.num_queues); + if (err) { + netif_err(priv, drv, priv->dev, + "failed to destroy tx queues\n"); + /* This failure will trigger a reset - no need to clean up */ + return err; } - for (i = 0; i < priv->rx_cfg.num_queues; i++) { - err = gve_adminq_destroy_rx_queue(priv, i); - if (err) { - netif_err(priv, drv, priv->dev, - "failed to destroy rx queue %d\n", - i); - /* This failure will trigger a reset - no need to clean - * up - */ - return err; - } - netif_dbg(priv, drv, priv->dev, "destroyed rx queue %d\n", i); + netif_dbg(priv, drv, priv->dev, "destroyed tx queues\n"); + err = gve_adminq_destroy_rx_queues(priv, priv->rx_cfg.num_queues); + if (err) { + netif_err(priv, drv, priv->dev, + "failed to destroy rx queues\n"); + /* This failure will trigger a reset - no need to clean up */ + return err; } + netif_dbg(priv, drv, priv->dev, "destroyed rx queues\n"); return 0; } From patchwork Tue Sep 1 21:51:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Awogbemila X-Patchwork-Id: 1355425 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20161025 header.b=DggOQ4gg; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4Bh15C5YKKz9sR4 for ; Wed, 2 Sep 2020 07:52:31 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729747AbgIAVwZ (ORCPT ); Tue, 1 Sep 2020 17:52:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56802 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729501AbgIAVwL (ORCPT ); Tue, 1 Sep 2020 17:52:11 -0400 Received: from mail-pf1-x449.google.com (mail-pf1-x449.google.com [IPv6:2607:f8b0:4864:20::449]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 34C60C061247 for ; Tue, 1 Sep 2020 14:52:11 -0700 (PDT) Received: by mail-pf1-x449.google.com with SMTP id f5so1491118pfe.2 for ; Tue, 01 Sep 2020 14:52:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=qm0KYWWyWC7VzRV6b5FGuhbeKRRJgNfVwQb8pLTqbjs=; b=DggOQ4ggeg+x8H/uwwfXk16LN9Yw44jHKwRnD7VbwrileYaaRPM7E+Lmj4/6vs+Snp mmqX2HfNrj+PDc/eZDsk9qP61+RzfXn0aS9Oj5b2QgCAL90J7p713QfoUGkLDzYRYxs5 p0BFWXHDlPDF1sv3mTniOGPcubbCgkPBi19dHk8jWzKNmuFeYa24ic36p3yDR61QUl6U ekN2pAG16qujUL2jbBVnOx9oOODIQPBqV9lV5hJl63ZxutnwqTQKE9djOkqCFyyaerNP YwWi8Fkg8q7+BbaeG/5mc6i6a4Ymt+LK7b9TZ5W5FfEJpS3c5oXpNJbddLUK31ybz8Kp 090w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=qm0KYWWyWC7VzRV6b5FGuhbeKRRJgNfVwQb8pLTqbjs=; b=lk322PIpEFjqEyKWygSBo/XJC1ili+ZbLynkMob+Hvb+CT4XsLoSjRYeu8KlXOITk1 m3T0w+dmup0lww3/KeR0wZJYqMcZnKtXtJ0+prxvwbqCuLnZOUF5PqD9rUZzgLnRL/y/ n0CAvSRdrCUai/26Kag8hXjbC2Suw5uTXQmz2We5ziuVUgb9MFIG5wqsv5CXjA6RUyNf xAjU19c4RcPj4ocSGKWUNf0d2oqbolEgCy3wJnUqLzxZ1HnAq08sr6O5JJgrqrxpBZao i/LhacXkGhAObKkhTiYRO/kuPGPhz9e3Dl8uzdXX34fAZjy7wivp5bOfeejX1D0pcTac njxQ== X-Gm-Message-State: AOAM533666iu1n4Cx2vM/7hTYlDQ7gNwdP8tJASVbFr3lNWU6RHNJEya es/BgewZHnTf9ReU6ktuv0BB3ezGGU+4UAI+2JRiNGCgPOs0eoYLtQaXLF4z1yqjFbUsvBlHDet iZpO9/oKBl7tnu4QLAcy8EVzg76M/QBtBscLC4fhfebA5FxRyrO76kMRRP3Bis/lLY8nNf6jv X-Google-Smtp-Source: ABdhPJzLlQds8V7c1NvsaWrSE5ZVzuQh4mq0fBB9d4l2aT+NfGMRze1H44pzmJl/8sPADXtyw++4eExzrrEIzyRP X-Received: from awogbemila.sea.corp.google.com ([2620:15c:100:202:1ea0:b8ff:fe73:6cc0]) (user=awogbemila job=sendgmr) by 2002:a17:902:7e82:b029:cf:85aa:69f6 with SMTP id z2-20020a1709027e82b02900cf85aa69f6mr807202pla.3.1598997130552; Tue, 01 Sep 2020 14:52:10 -0700 (PDT) Date: Tue, 1 Sep 2020 14:51:48 -0700 In-Reply-To: <20200901215149.2685117-1-awogbemila@google.com> Message-Id: <20200901215149.2685117-9-awogbemila@google.com> Mime-Version: 1.0 References: <20200901215149.2685117-1-awogbemila@google.com> X-Mailer: git-send-email 2.28.0.402.g5ffc5be6b7-goog Subject: [PATCH net-next v2 8/9] gve: Use link status register to report link status From: David Awogbemila To: netdev@vger.kernel.org Cc: Patricio Noyola , Yangchun Fu , David Awogbemila Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Patricio Noyola This makes the driver better aware of the connectivity status of the device. Based on the device's status register, the driver can call netif_carrier_{on,off}. Reviewed-by: Yangchun Fu Signed-off-by: Patricio Noyola Signed-off-by: David Awogbemila --- drivers/net/ethernet/google/gve/gve_main.c | 24 +++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/google/gve/gve_main.c b/drivers/net/ethernet/google/gve/gve_main.c index d5019132874d..4ddcc1836d1c 100644 --- a/drivers/net/ethernet/google/gve/gve_main.c +++ b/drivers/net/ethernet/google/gve/gve_main.c @@ -769,7 +769,7 @@ static int gve_open(struct net_device *dev) gve_set_device_rings_ok(priv); gve_turnup(priv); - netif_carrier_on(dev); + queue_work(priv->gve_wq, &priv->service_task); priv->interface_up_cnt++; return 0; @@ -1026,16 +1026,34 @@ void gve_handle_report_stats(struct gve_priv *priv) } } +static void gve_handle_link_status(struct gve_priv *priv, bool link_status) +{ + if (!gve_get_napi_enabled(priv)) + return; + + if (link_status == netif_carrier_ok(priv->dev)) + return; + + if (link_status) { + dev_info(&priv->pdev->dev, "Device link is up.\n"); + netif_carrier_on(priv->dev); + } else { + dev_info(&priv->pdev->dev, "Device link is down.\n"); + netif_carrier_off(priv->dev); + } +} + /* Handle NIC status register changes, reset requests and report stats */ static void gve_service_task(struct work_struct *work) { struct gve_priv *priv = container_of(work, struct gve_priv, service_task); + u32 status = ioread32be(&priv->reg_bar0->device_status); - gve_handle_status(priv, - ioread32be(&priv->reg_bar0->device_status)); + gve_handle_status(priv, status); gve_handle_reset(priv); + gve_handle_link_status(priv, GVE_DEVICE_STATUS_LINK_STATUS_MASK & status); if (gve_get_do_report_stats(priv)) { gve_handle_report_stats(priv); gve_clear_do_report_stats(priv); From patchwork Tue Sep 1 21:51:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Awogbemila X-Patchwork-Id: 1355424 X-Patchwork-Delegate: davem@davemloft.net Return-Path: X-Original-To: patchwork-incoming-netdev@ozlabs.org Delivered-To: patchwork-incoming-netdev@ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=vger.kernel.org (client-ip=23.128.96.18; helo=vger.kernel.org; envelope-from=netdev-owner@vger.kernel.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20161025 header.b=YQ8ryqvG; dkim-atps=neutral Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by ozlabs.org (Postfix) with ESMTP id 4Bh1580B7wz9sR4 for ; Wed, 2 Sep 2020 07:52:28 +1000 (AEST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729840AbgIAVw0 (ORCPT ); Tue, 1 Sep 2020 17:52:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56810 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729520AbgIAVwN (ORCPT ); Tue, 1 Sep 2020 17:52:13 -0400 Received: from mail-pl1-x649.google.com (mail-pl1-x649.google.com [IPv6:2607:f8b0:4864:20::649]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0881FC061244 for ; Tue, 1 Sep 2020 14:52:13 -0700 (PDT) Received: by mail-pl1-x649.google.com with SMTP id y13so1311279plr.1 for ; Tue, 01 Sep 2020 14:52:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=CZOU5zNBV7YhreEpl/ZEvsx5azVAeeK7Om2FNRiEDWc=; b=YQ8ryqvG4mlgnTuOKdOdPXJiTFgnoYN/VnEpah/ALSxc2cpBcRZ/TJ8sh/MODzz50n 6gAlb4xtoATrwCMvTpJRmksBCO6h+px6J/+kmv3wIAPpaeskfpxAVoumjccEpQsm7C6e 13NVTr7FAsFuUyFDwsdyN2e+17+rY5+bKjXyF54W7qdQxFIwwp2fXXAzGFFjBP0u8val 8uo3m0Nsl8dU9vjHPL6ar8PDo7h7d8qsI0HZq5EQmWaQSy+P4IpJ1EhOtLqpnjSWDSI9 I8azsIzhd7b2RBnrjXUlN88mh2P4CIxknTsWenPqn1phJKv+SDuR26IJrPfcvG6H7sG+ HIDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=CZOU5zNBV7YhreEpl/ZEvsx5azVAeeK7Om2FNRiEDWc=; b=lZuJMeV+sa5oX7wfkPeiwk1raFnf+csBq9jXqC9z/E1EcxIXLcjN3s+Nsot/gaHWwZ 9bZVeyO9XBlHCGhTB8TIk+XxBjBrcvGBtARdhw4FGI6LoYnypjj3QXJPL6By4WkLpabJ ANIRuzrKd7s/htBawa0O1R9+HrkVouAOPxGA0iH6uU4chpurW61uGJHx2j8UoH3j0AEh ya3fhuCryjp+5igsnbP1jPuIG8s/9ARxJRw2RpzB2DT7VxoYgm1EhPK8Y/OHT0sQU2Ea bUoaGOhm0GnPVGAEf3Mwe6+Lwtny85k2WDiwxm+hy1Z+kHL0lT0PVo6fnZ6UD6W/xLvc II1g== X-Gm-Message-State: AOAM533lfezSeY9NCE5lCHjSuRoz/zo0TGPTQJLc75fHk/Vjudpd1cZ/ ZzMfFvgwTPoPKI62Ro8HIrDUkT/3wqTJo2JAAdkOCp99RH/BkCUk62brYmh/hKCwmamJzSUeEEx nxpPYiyjwrS3wkCwZPbJvhqc3XVDyjzCJOW14Cb/3Oh/Xum5KzTnUAwqHfrU75OnGSSy4bPEK X-Google-Smtp-Source: ABdhPJytCp2/XLOMlTBB0VJ/Wj4TJc3zYWe3phfQndPbuJOkHSjFr8IdVzPN8Ik+AU5QOgCKwVpkgkPpH0/SxxXc X-Received: from awogbemila.sea.corp.google.com ([2620:15c:100:202:1ea0:b8ff:fe73:6cc0]) (user=awogbemila job=sendgmr) by 2002:a17:90b:4b89:: with SMTP id lr9mr3542807pjb.126.1598997132456; Tue, 01 Sep 2020 14:52:12 -0700 (PDT) Date: Tue, 1 Sep 2020 14:51:49 -0700 In-Reply-To: <20200901215149.2685117-1-awogbemila@google.com> Message-Id: <20200901215149.2685117-10-awogbemila@google.com> Mime-Version: 1.0 References: <20200901215149.2685117-1-awogbemila@google.com> X-Mailer: git-send-email 2.28.0.402.g5ffc5be6b7-goog Subject: [PATCH net-next v2 9/9] gve: Enable Link Speed Reporting in the driver. From: David Awogbemila To: netdev@vger.kernel.org Cc: David Awogbemila , Yangchun Fu Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org This change allows the driver to report the device link speed when the ethtool command: ethtool is run. Getting the link speed is done via a new admin queue command: ReportLinkSpeed. Reviewed-by: Yangchun Fu Signed-off-by: David Awogbemila --- drivers/net/ethernet/google/gve/gve.h | 4 +++ drivers/net/ethernet/google/gve/gve_adminq.c | 31 +++++++++++++++++++ drivers/net/ethernet/google/gve/gve_adminq.h | 9 ++++++ drivers/net/ethernet/google/gve/gve_ethtool.c | 14 ++++++++- 4 files changed, 57 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/google/gve/gve.h b/drivers/net/ethernet/google/gve/gve.h index bc54059f9b2e..4a8d77013ebe 100644 --- a/drivers/net/ethernet/google/gve/gve.h +++ b/drivers/net/ethernet/google/gve/gve.h @@ -232,6 +232,7 @@ struct gve_priv { u32 adminq_dcfg_device_resources_cnt; u32 adminq_set_driver_parameter_cnt; u32 adminq_report_stats_cnt; + u32 adminq_report_link_speed_cnt; /* Global stats */ u32 interface_up_cnt; /* count of times interface turned up since last reset */ @@ -252,6 +253,9 @@ struct gve_priv { unsigned long service_timer_period; struct timer_list service_timer; + /* Gvnic device link speed from hypervisor. */ + u64 link_speed; + /* Gvnic device's dma mask, set during probe. */ u8 dma_mask; }; diff --git a/drivers/net/ethernet/google/gve/gve_adminq.c b/drivers/net/ethernet/google/gve/gve_adminq.c index 341a17b36f06..2372e18943b8 100644 --- a/drivers/net/ethernet/google/gve/gve_adminq.c +++ b/drivers/net/ethernet/google/gve/gve_adminq.c @@ -36,6 +36,7 @@ int gve_adminq_alloc(struct device *dev, struct gve_priv *priv) priv->adminq_dcfg_device_resources_cnt = 0; priv->adminq_set_driver_parameter_cnt = 0; priv->adminq_report_stats_cnt = 0; + priv->adminq_report_link_speed_cnt = 0; /* Setup Admin queue with the device */ iowrite32be(priv->adminq_bus_addr / PAGE_SIZE, @@ -238,6 +239,9 @@ static int gve_adminq_issue_cmd(struct gve_priv *priv, case GVE_ADMINQ_REPORT_STATS: priv->adminq_report_stats_cnt++; break; + case GVE_ADMINQ_REPORT_LINK_SPEED: + priv->adminq_report_link_speed_cnt++; + break; default: dev_err(&priv->pdev->dev, "unknown AQ command opcode %d\n", opcode); } @@ -596,3 +600,30 @@ int gve_adminq_report_stats(struct gve_priv *priv, u64 stats_report_len, return gve_adminq_execute_cmd(priv, &cmd); } +int gve_adminq_report_link_speed(struct gve_priv *priv) +{ + union gve_adminq_command gvnic_cmd; + dma_addr_t link_speed_region_bus; + __be64 *link_speed_region; + int err; + + link_speed_region = + dma_alloc_coherent(&priv->pdev->dev, sizeof(*link_speed_region), + &link_speed_region_bus, GFP_KERNEL); + + if (!link_speed_region) + return -ENOMEM; + + memset(&gvnic_cmd, 0, sizeof(gvnic_cmd)); + gvnic_cmd.opcode = cpu_to_be32(GVE_ADMINQ_REPORT_LINK_SPEED); + gvnic_cmd.report_link_speed.link_speed_address = + cpu_to_be64(link_speed_region_bus); + + err = gve_adminq_execute_cmd(priv, &gvnic_cmd); + + priv->link_speed = be64_to_cpu(*link_speed_region); + dma_free_coherent(&priv->pdev->dev, sizeof(*link_speed_region), link_speed_region, + link_speed_region_bus); + return err; +} + diff --git a/drivers/net/ethernet/google/gve/gve_adminq.h b/drivers/net/ethernet/google/gve/gve_adminq.h index 784830f75b7c..281de8326bc5 100644 --- a/drivers/net/ethernet/google/gve/gve_adminq.h +++ b/drivers/net/ethernet/google/gve/gve_adminq.h @@ -22,6 +22,7 @@ enum gve_adminq_opcodes { GVE_ADMINQ_DECONFIGURE_DEVICE_RESOURCES = 0x9, GVE_ADMINQ_SET_DRIVER_PARAMETER = 0xB, GVE_ADMINQ_REPORT_STATS = 0xC, + GVE_ADMINQ_REPORT_LINK_SPEED = 0xD }; /* Admin queue status codes */ @@ -181,6 +182,12 @@ struct gve_adminq_report_stats { static_assert(sizeof(struct gve_adminq_report_stats) == 24); +struct gve_adminq_report_link_speed { + __be64 link_speed_address; +}; + +static_assert(sizeof(struct gve_adminq_report_link_speed) == 8); + struct stats { __be32 stat_name; __be32 queue_id; @@ -228,6 +235,7 @@ union gve_adminq_command { struct gve_adminq_unregister_page_list unreg_page_list; struct gve_adminq_set_driver_parameter set_driver_param; struct gve_adminq_report_stats report_stats; + struct gve_adminq_report_link_speed report_link_speed; }; }; u8 reserved[64]; @@ -255,4 +263,5 @@ int gve_adminq_unregister_page_list(struct gve_priv *priv, u32 page_list_id); int gve_adminq_set_mtu(struct gve_priv *priv, u64 mtu); int gve_adminq_report_stats(struct gve_priv *priv, u64 stats_report_len, dma_addr_t stats_report_addr, u64 interval); +int gve_adminq_report_link_speed(struct gve_priv *priv); #endif /* _GVE_ADMINQ_H */ diff --git a/drivers/net/ethernet/google/gve/gve_ethtool.c b/drivers/net/ethernet/google/gve/gve_ethtool.c index db5642fe966f..4830c873cd78 100644 --- a/drivers/net/ethernet/google/gve/gve_ethtool.c +++ b/drivers/net/ethernet/google/gve/gve_ethtool.c @@ -59,7 +59,7 @@ static const char gve_gstrings_adminq_stats[][ETH_GSTRING_LEN] = { "adminq_create_tx_queue_cnt", "adminq_create_rx_queue_cnt", "adminq_destroy_tx_queue_cnt", "adminq_destroy_rx_queue_cnt", "adminq_dcfg_device_resources_cnt", "adminq_set_driver_parameter_cnt", - "adminq_report_stats_cnt", + "adminq_report_stats_cnt", "adminq_report_link_speed_cnt" }; static const char gve_gstrings_priv_flags[][ETH_GSTRING_LEN] = { @@ -354,6 +354,7 @@ gve_get_ethtool_stats(struct net_device *netdev, data[i++] = priv->adminq_dcfg_device_resources_cnt; data[i++] = priv->adminq_set_driver_parameter_cnt; data[i++] = priv->adminq_report_stats_cnt; + data[i++] = priv->adminq_report_link_speed_cnt; } static void gve_get_channels(struct net_device *netdev, @@ -506,6 +507,16 @@ static int gve_set_priv_flags(struct net_device *netdev, u32 flags) return 0; } +static int gve_get_link_ksettings(struct net_device *netdev, + struct ethtool_link_ksettings *cmd) +{ + struct gve_priv *priv = netdev_priv(netdev); + int err = gve_adminq_report_link_speed(priv); + + cmd->base.speed = priv->link_speed; + return err; +} + const struct ethtool_ops gve_ethtool_ops = { .get_drvinfo = gve_get_drvinfo, .get_strings = gve_get_strings, @@ -522,4 +533,5 @@ const struct ethtool_ops gve_ethtool_ops = { .set_tunable = gve_set_tunable, .get_priv_flags = gve_get_priv_flags, .set_priv_flags = gve_set_priv_flags, + .get_link_ksettings = gve_get_link_ksettings };