From patchwork Wed Aug 17 08:51:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gerald Yang X-Patchwork-Id: 1667154 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=MWLny4ZR; dkim-atps=neutral Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4M71xH6ZhBz1ygF for ; Wed, 17 Aug 2022 18:52:35 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1oOEmm-0000ku-E7; Wed, 17 Aug 2022 08:52:24 +0000 Received: from smtp-relay-internal-1.internal ([10.131.114.114] helo=smtp-relay-internal-1.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1oOEmX-0000Or-Hw for kernel-team@lists.ubuntu.com; Wed, 17 Aug 2022 08:52:09 +0000 Received: from mail-pf1-f199.google.com (mail-pf1-f199.google.com [209.85.210.199]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id 652313F46B for ; Wed, 17 Aug 2022 08:52:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1660726327; bh=UcGTP2GKqSjG53nObBxMBrKZPQ2F0MzRYmNASivPigw=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=MWLny4ZRxlwJU+PS6f5pkHR5MdeA53ME0n03KHaC0mYf60vhZhjnotdyiBKGhHBFC dFsdvuMgtsJEaZaNvCuA3H2xFYnoO7kajFRFEMfEWi5INmwabPBTtUNotd30g9Yeq2 oMq329lp4b+VPFM4qJj/UC6kmRxYCsGVjKwhjjW4Z/amzCsfNX2gKz6MnsHKmAWbIZ sJuFbeP3Jq6Xut4OovcK7jbb+VOwnsZVwR1JEpXAyrPs2fSZrX2ye5fUSRQetcnuUd cx9iMbLdrpihMv5nBzn2U5Y7qqVfCVJjs8BQhOZDgManMimfsUuCFpPElH1W4c+mXx yGFuoljkNMHJQ== Received: by mail-pf1-f199.google.com with SMTP id y8-20020aa79428000000b0052f17a0627cso4733325pfo.1 for ; Wed, 17 Aug 2022 01:52:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc; bh=UcGTP2GKqSjG53nObBxMBrKZPQ2F0MzRYmNASivPigw=; b=Kg20teeh8N8B8cvhFJddof4iJmtXkVCt9qALM2PneFI02EbBAAwLbGiAZqR/f2A8Fq 3oZ5vLH/mli4qCZZvj1Choyiy/y/W2I4asbgBb6xv35GNav25+K3vTHC5aJavVTmKYCr N2ViOvd/ysa1hFlBJXIZXjqUHhNAedXnOPxmiU5AiwOwiX/SV4paGNmCxvjGV4a/xlBR tjIC1U1yxH06/ehlXMxKBhYEPFS0KDIqFnXmhGV86xtbosC4VKV6xfcFjOOk4BxKA6ck ISXSVaHkaEt4O6nbmBOI/kaTGa04oHGE9itgfYyPl/rwt7UWBXnResTIyblbSUMhtkln 7fFg== X-Gm-Message-State: ACgBeo02igiqirEzk1Z7eMfCxGohivmeDZaqK1r3vmPiy6gpuz+BgNBO Q6DvgCgZrY24APT52+Oucc0F/i1pyxdmybapc96HlITcOafFtPivipLk7RgW1+NiiIFORlPqdQ+ SjoJmBhK1JPWTTDyDL97J2/dxma/7zYd9xVSX2AzHSg== X-Received: by 2002:a17:902:ce11:b0:172:6f2c:a910 with SMTP id k17-20020a170902ce1100b001726f2ca910mr13920398plg.156.1660726325665; Wed, 17 Aug 2022 01:52:05 -0700 (PDT) X-Google-Smtp-Source: AA6agR6RFXJEUSkVyr1BrQhC5hAO9xW+FtO2d5Yn2lOqI98ge93KgTDs96S7w96zzFkmJ6l3lJTXpw== X-Received: by 2002:a17:902:ce11:b0:172:6f2c:a910 with SMTP id k17-20020a170902ce1100b001726f2ca910mr13920371plg.156.1660726325158; Wed, 17 Aug 2022 01:52:05 -0700 (PDT) Received: from localhost.localdomain (220-135-31-21.hinet-ip.hinet.net. [220.135.31.21]) by smtp.gmail.com with ESMTPSA id s90-20020a17090a69e300b001f522180d46sm1001033pjj.8.2022.08.17.01.52.04 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Aug 2022 01:52:04 -0700 (PDT) From: Gerald Yang To: kernel-team@lists.ubuntu.com Subject: [SRU][jammy/linux-aws][PATCH 07/20] UBUNTU: SAUCE: xen-netfront: add callbacks for PM suspend and hibernation support Date: Wed, 17 Aug 2022 16:51:35 +0800 Message-Id: <20220817085150.2078055-10-gerald.yang@canonical.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220817085150.2078055-1-gerald.yang@canonical.com> References: <20220817085150.2078055-1-gerald.yang@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Munehisa Kamata BugLink: https://bugs.launchpad.net/bugs/1968062 Add freeze and restore callbacks for PM suspend and hibernation support. The freeze handler simply disconnects the frotnend from the backend and frees resources associated with queues after disabling the net_device from the system. The restore handler just changes the frontend state and let the xenbus handler to re-allocate the resources and re-connect to the backend. This can be performed transparently to the rest of the system. The handlers are used for both PM suspend and hibernation so that we can keep the existing suspend/resume callbacks for Xen suspend without modification. Freezing netfront devices is normally expected to finish within a few hundred milliseconds, but it can rarely take more than 5 seconds and hit the hard coded timeout, it would depend on backend state which may be congested and/or have complex configuration. While it's rare case, longer default timeout seems a bit more reasonable here to avoid hitting the timeout. Also, make it configurable via module parameter so that we can cover broader setups than what we know currently. Signed-off-by: Munehisa Kamata Signed-off-by: Anchal Agarwal Reviewed-by: Eduardo Valentin Reviewed-by: Munehisa Kamata CR: https://cr.amazon.com/r/8297632 (cherry picked from commit 78b1b3b4dc215e8046806de5e8e1fa4e7ffe5fd2 amazon-5.15.y/mainline) Signed-off-by: Gerald Yang Signed-off-by: Matthew Ruffell --- drivers/net/xen-netfront.c | 97 +++++++++++++++++++++++++++++++++++++- 1 file changed, 96 insertions(+), 1 deletion(-) diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c index 2492a27467b4..0304fa9f40dc 100644 --- a/drivers/net/xen-netfront.c +++ b/drivers/net/xen-netfront.c @@ -43,6 +43,7 @@ #include #include #include +#include #include #include #include @@ -59,6 +60,12 @@ #include #include +enum netif_freeze_state { + NETIF_FREEZE_STATE_UNFROZEN, + NETIF_FREEZE_STATE_FREEZING, + NETIF_FREEZE_STATE_FROZEN, +}; + /* Module parameters */ #define MAX_QUEUES_DEFAULT 8 static unsigned int xennet_max_queues; @@ -68,6 +75,12 @@ MODULE_PARM_DESC(max_queues, #define XENNET_TIMEOUT (5 * HZ) +static unsigned int netfront_freeze_timeout_secs = 10; +module_param_named(freeze_timeout_secs, + netfront_freeze_timeout_secs, uint, 0644); +MODULE_PARM_DESC(freeze_timeout_secs, + "timeout when freezing netfront device in seconds"); + static const struct ethtool_ops xennet_ethtool_ops; struct netfront_cb { @@ -176,6 +189,10 @@ struct netfront_info { bool broken; atomic_t rx_gso_checksum_fixup; + + int freeze_state; + + struct completion wait_backend_disconnected; }; struct netfront_rx_info { @@ -872,6 +889,21 @@ static void xennet_set_rx_rsp_cons(struct netfront_queue *queue, RING_IDX val) spin_unlock_irqrestore(&queue->rx_cons_lock, flags); } +static int xennet_disable_interrupts(struct net_device *dev) +{ + struct netfront_info *np = netdev_priv(dev); + unsigned int num_queues = dev->real_num_tx_queues; + unsigned int i; + struct netfront_queue *queue; + + for (i = 0; i < num_queues; ++i) { + queue = &np->queues[i]; + disable_irq(queue->tx_irq); + disable_irq(queue->rx_irq); + } + return 0; +} + static void xennet_move_rx_slot(struct netfront_queue *queue, struct sk_buff *skb, grant_ref_t ref) { @@ -1681,6 +1713,8 @@ static struct net_device *xennet_create_dev(struct xenbus_device *dev) np->queues = NULL; + init_completion(&np->wait_backend_disconnected); + err = -ENOMEM; np->rx_stats = netdev_alloc_pcpu_stats(struct netfront_stats); if (np->rx_stats == NULL) @@ -2231,6 +2265,50 @@ static int xennet_create_queues(struct netfront_info *info, return 0; } +static int netfront_freeze(struct xenbus_device *dev) +{ + struct netfront_info *info = dev_get_drvdata(&dev->dev); + unsigned long timeout = netfront_freeze_timeout_secs * HZ; + int err = 0; + + xennet_disable_interrupts(info->netdev); + + netif_device_detach(info->netdev); + + info->freeze_state = NETIF_FREEZE_STATE_FREEZING; + + /* Kick the backend to disconnect */ + xenbus_switch_state(dev, XenbusStateClosing); + + /* We don't want to move forward before the frontend is diconnected + * from the backend cleanly. + */ + timeout = wait_for_completion_timeout(&info->wait_backend_disconnected, + timeout); + if (!timeout) { + err = -EBUSY; + xenbus_dev_error(dev, err, "Freezing timed out;" + "the device may become inconsistent state"); + return err; + } + + /* Tear down queues */ + xennet_disconnect_backend(info); + xennet_destroy_queues(info); + + info->freeze_state = NETIF_FREEZE_STATE_FROZEN; + + return err; +} + +static int netfront_restore(struct xenbus_device *dev) +{ + /* Kick the backend to re-connect */ + xenbus_switch_state(dev, XenbusStateInitialising); + + return 0; +} + /* Common code used when first setting up, and when resuming. */ static int talk_to_netback(struct xenbus_device *dev, struct netfront_info *info) @@ -2456,6 +2534,8 @@ static int xennet_connect(struct net_device *dev) spin_unlock_bh(&queue->rx_lock); } + np->freeze_state = NETIF_FREEZE_STATE_UNFROZEN; + return 0; } @@ -2493,10 +2573,22 @@ static void netback_changed(struct xenbus_device *dev, break; case XenbusStateClosed: - if (dev->state == XenbusStateClosed) + if (dev->state == XenbusStateClosed) { + /* dpm context is waiting for the backend */ + if (np->freeze_state == NETIF_FREEZE_STATE_FREEZING) + complete(&np->wait_backend_disconnected); break; + } fallthrough; /* Missed the backend's CLOSING state */ case XenbusStateClosing: + /* We may see unexpected Closed or Closing from the backend. + * Just ignore it not to prevent the frontend from being + * re-connected in the case of PM suspend or hibernation. + */ + if (np->freeze_state == NETIF_FREEZE_STATE_FROZEN && + dev->state == XenbusStateInitialising) { + break; + } xenbus_frontend_closed(dev); break; } @@ -2658,6 +2750,9 @@ static struct xenbus_driver netfront_driver = { .probe = netfront_probe, .remove = xennet_remove, .resume = netfront_resume, + .freeze = netfront_freeze, + .thaw = netfront_restore, + .restore = netfront_restore, .otherend_changed = netback_changed, };