From patchwork Wed Aug 17 08:51:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gerald Yang X-Patchwork-Id: 1667155 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=TmUy3rkb; dkim-atps=neutral Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4M71xQ07mrz1ygF for ; Wed, 17 Aug 2022 18:52:42 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1oOEmt-0000xa-DJ; Wed, 17 Aug 2022 08:52:31 +0000 Received: from smtp-relay-internal-0.internal ([10.131.114.225] helo=smtp-relay-internal-0.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1oOEmY-0000Ru-PA for kernel-team@lists.ubuntu.com; Wed, 17 Aug 2022 08:52:10 +0000 Received: from mail-pj1-f70.google.com (mail-pj1-f70.google.com [209.85.216.70]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 5B2BD3FB94 for ; Wed, 17 Aug 2022 08:52:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1660726329; bh=7rqBGZgkKCPWTbp255zbWWiq3ePWTX5K5C7JCgVBCrI=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=TmUy3rkbKZsmHgllyf/t/x8GHGsrg09aK/gO3cvJTg3TKk2OdWCM5Er4X0hEomWoa T8+k26o6cPIPKVGR/QXWUFEcG+C60eb2x2A1Eno+GS1/cdrrcBNMO7G0/xc0MRM3TJ NGUf950TzARA/rJTu8wCMBQx8k6VCCazQFEp8O7H4yrqcgDxBkrnL3jSbTKn2JWeXB QRZYcqe9kmUReKgbbUUgv+lylFq8pcDxeP1ePg6xJS9+KkyvEEZwZVtJV6reIEnEvJ +fanUswaL4B0gJCOsMacE4DH3AId6hF6dWSCYf9tQbE/m6wMlUO1tyCDCqhA8lJ0vf c1PrmboddAC8w== Received: by mail-pj1-f70.google.com with SMTP id q6-20020a17090a1b0600b001f558bbb924so646244pjq.3 for ; Wed, 17 Aug 2022 01:52:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc; bh=7rqBGZgkKCPWTbp255zbWWiq3ePWTX5K5C7JCgVBCrI=; b=vWcxux3tMb/4nT7VPC7S6uX/4BsWRxz/BAwnpvfPq/tHp+Ux8CbPFwkG8LAwZMALl+ g8GjVbTjuGRZy6XoTLZEC0sMbAUF2XPLekg488mLnr/1vzl+QYlawXeXIl1RlqW6zdu+ MYxoVHfKUcqjmwpiDIL5ZkPbF/VQmbI3/CW7uYI/P8dpiVpwztueXX+4SnjrZYttAUzt p6G9yH6VBfDp+Wrvg22YcwOIZfkC9BjjszZ40K09FL1N6uGc/eFWxllMoCARBFNN4lGJ vhZT2X1PFfjn9h9VI1Udnvt2/WKDFeF7NnJBtRo4oNgy2QvUFUZrU8QyDB+GOtQ8XrZF glGg== X-Gm-Message-State: ACgBeo02sY5aSFXRWF6COG3u5UoKR09luZAPbNtZrNqsq/p1BiU+3V9h iyfyCXP6BCG+JrZ1kCU8IDIywqWOYX3AcEsoJoDvzE5Il1L86L6GLu6ykZ7NdgcXp0iZgNitvtn BqVk+kB29wFx4AIjXJBQHEu9zPhJvROD/PNOEuMNhxw== X-Received: by 2002:a17:903:285:b0:16e:dcb7:937a with SMTP id j5-20020a170903028500b0016edcb7937amr25018734plr.4.1660726327793; Wed, 17 Aug 2022 01:52:07 -0700 (PDT) X-Google-Smtp-Source: AA6agR4d7vU/k9MM6w+6vOTEVEeOoVZm9tKBIjn+Hd91TSkuZMGvsyIrwKhLsiuOHtEOXtQ+mjZR3A== X-Received: by 2002:a17:903:285:b0:16e:dcb7:937a with SMTP id j5-20020a170903028500b0016edcb7937amr25018712plr.4.1660726327369; Wed, 17 Aug 2022 01:52:07 -0700 (PDT) Received: from localhost.localdomain (220-135-31-21.hinet-ip.hinet.net. [220.135.31.21]) by smtp.gmail.com with ESMTPSA id s90-20020a17090a69e300b001f522180d46sm1001033pjj.8.2022.08.17.01.52.06 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 Aug 2022 01:52:07 -0700 (PDT) From: Gerald Yang To: kernel-team@lists.ubuntu.com Subject: [SRU][jammy/linux-aws][kinetic/linux-aws][PATCH 08/20] UBUNTU: SAUCE: xen/time: introduce xen_{save, restore}_steal_clock Date: Wed, 17 Aug 2022 16:51:37 +0800 Message-Id: <20220817085150.2078055-12-gerald.yang@canonical.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220817085150.2078055-1-gerald.yang@canonical.com> References: <20220817085150.2078055-1-gerald.yang@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Munehisa Kamata BugLink: https://bugs.launchpad.net/bugs/1968062 Currently, steal time accounting code in scheduler expects steal clock callback to provide monotonically increasing value. If the accounting code receives a smaller value than previous one, it uses a negative value to calculate steal time and results in incorrectly updated idle and steal time accounting. This breaks userspace tools which read /proc/stat. top - 08:05:35 up 2:12, 3 users, load average: 0.00, 0.07, 0.23 Tasks: 80 total, 1 running, 79 sleeping, 0 stopped, 0 zombie Cpu(s): 0.0%us, 0.0%sy, 0.0%ni,30100.0%id, 0.0%wa, 0.0%hi, 0.0%si,-1253874204672.0%st This can actually happen when a Xen PVHVM guest gets restored from hibernation, because such a restored guest is just a fresh domain from Xen perspective and the time information in runstate info starts over from scratch. This patch introduces xen_save_steal_clock() which saves current values in runstate info into per-cpu variables. Its couterpart, xen_restore_steal_clock(), sets offset if it found the current values in runstate info are smaller than previous ones. xen_steal_clock() is also modified to use the offset to ensure that scheduler only sees monotonically increasing number. Signed-off-by: Munehisa Kamata Signed-off-by: Anchal Agarwal Reviewed-by: Munehisa Kamata Reviewed-by: Eduardo Valentin CR: https://cr.amazon.com/r/8273209/ (cherry picked from commit 7a4398155fdabe4e9ad27561e44fb6f6bc9d595e amazon-5.15.y/mainline) Signed-off-by: Gerald Yang Signed-off-by: Matthew Ruffell --- drivers/xen/time.c | 29 ++++++++++++++++++++++++++++- include/xen/xen-ops.h | 2 ++ 2 files changed, 30 insertions(+), 1 deletion(-) diff --git a/drivers/xen/time.c b/drivers/xen/time.c index 152dd33bb223..bf41e5cf1332 100644 --- a/drivers/xen/time.c +++ b/drivers/xen/time.c @@ -24,6 +24,9 @@ static DEFINE_PER_CPU(struct vcpu_runstate_info, xen_runstate); static DEFINE_PER_CPU(u64[4], old_runstate_time); +static DEFINE_PER_CPU(u64, xen_prev_steal_clock); +static DEFINE_PER_CPU(u64, xen_steal_clock_offset); + /* return an consistent snapshot of 64-bit time/counter value */ static u64 get64(const u64 *p) { @@ -150,7 +153,7 @@ bool xen_vcpu_stolen(int vcpu) return per_cpu(xen_runstate, vcpu).state == RUNSTATE_runnable; } -u64 xen_steal_clock(int cpu) +static u64 __xen_steal_clock(int cpu) { struct vcpu_runstate_info state; @@ -158,6 +161,30 @@ u64 xen_steal_clock(int cpu) return state.time[RUNSTATE_runnable] + state.time[RUNSTATE_offline]; } +u64 xen_steal_clock(int cpu) +{ + return __xen_steal_clock(cpu) + per_cpu(xen_steal_clock_offset, cpu); +} + +void xen_save_steal_clock(int cpu) +{ + per_cpu(xen_prev_steal_clock, cpu) = xen_steal_clock(cpu); +} + +void xen_restore_steal_clock(int cpu) +{ + u64 steal_clock = __xen_steal_clock(cpu); + + if (per_cpu(xen_prev_steal_clock, cpu) > steal_clock) { + /* Need to update the offset */ + per_cpu(xen_steal_clock_offset, cpu) = + per_cpu(xen_prev_steal_clock, cpu) - steal_clock; + } else { + /* Avoid unnecessary steal clock warp */ + per_cpu(xen_steal_clock_offset, cpu) = 0; + } +} + void xen_setup_runstate_info(int cpu) { struct vcpu_register_runstate_memory_area area; diff --git a/include/xen/xen-ops.h b/include/xen/xen-ops.h index 86b773f32ecd..5ec79b720aef 100644 --- a/include/xen/xen-ops.h +++ b/include/xen/xen-ops.h @@ -37,6 +37,8 @@ void xen_time_setup_guest(void); void xen_manage_runstate_time(int action); void xen_get_runstate_snapshot(struct vcpu_runstate_info *res); u64 xen_steal_clock(int cpu); +void xen_save_steal_clock(int cpu); +void xen_restore_steal_clock(int cpu); int xen_setup_shutdown_event(void);