From patchwork Sun Feb 6 02:27:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haren Myneni X-Patchwork-Id: 1588890 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=FMkSdWph; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=2404:9400:2:0:216:3eff:fee1:b9f1; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2404:9400:2:0:216:3eff:fee1:b9f1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4JrtTt5ZzDz9s1l for ; Sun, 6 Feb 2022 13:27:46 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4JrtTt2XC8z3bcn for ; Sun, 6 Feb 2022 13:27:46 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=FMkSdWph; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=haren@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=FMkSdWph; dkim-atps=neutral Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4JrtTc5VpQz2xCM for ; Sun, 6 Feb 2022 13:27:32 +1100 (AEDT) Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 2161k8t6020750; Sun, 6 Feb 2022 02:27:27 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=message-id : subject : from : to : cc : date : in-reply-to : references : content-type : mime-version : content-transfer-encoding; s=pp1; bh=vMXSAvW5VXjoeYURulerKMM5tw10hTTr1u8/RXMQGD8=; b=FMkSdWphH0UJvaZW4w8EbxvrjDPisVBJpWUbz+KkYnMudJyX+yfsvI1bVlkTnIjBNx9g ro9IxZLrtrUqullMIPAwzQpDvcE3Ono4Fn7BUd5RiMHTzrpJ1wL8J2+1eDbZHqOJR1z/ 1/JO5lhIMPO701FyXfD2VORNbn1+BGlGEKWWBMbj8724sm0JMNkb7aqDRYW0UdbKsrI8 EPOXUyPxxDE6LFghNd2/zi1KChxCVOrhRJ1nvgJ2uNe0qaahV7m9oLlDMtZzbjH6Ndnj Hc1Bs+w+t34s+qFBpJvy0d89DMa+pBrHciZbBX1uPmGZfwgkaZL3BgaFmtTHqQ/go7QS +A== Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 3e1huwenj5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 06 Feb 2022 02:27:26 +0000 Received: from m0098414.ppops.net (m0098414.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 2162RQXp009862; Sun, 6 Feb 2022 02:27:26 GMT Received: from ppma05wdc.us.ibm.com (1b.90.2fa9.ip4.static.sl-reverse.com [169.47.144.27]) by mx0b-001b2d01.pphosted.com with ESMTP id 3e1huwenj1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 06 Feb 2022 02:27:26 +0000 Received: from pps.filterd (ppma05wdc.us.ibm.com [127.0.0.1]) by ppma05wdc.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 2162MsDU008833; Sun, 6 Feb 2022 02:27:25 GMT Received: from b01cxnp23033.gho.pok.ibm.com (b01cxnp23033.gho.pok.ibm.com [9.57.198.28]) by ppma05wdc.us.ibm.com with ESMTP id 3e1gv9hkhf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 06 Feb 2022 02:27:25 +0000 Received: from b01ledav006.gho.pok.ibm.com (b01ledav006.gho.pok.ibm.com [9.57.199.111]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2162RMkg46989750 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sun, 6 Feb 2022 02:27:22 GMT Received: from b01ledav006.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9BFF8AC065; Sun, 6 Feb 2022 02:27:22 +0000 (GMT) Received: from b01ledav006.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CC837AC064; Sun, 6 Feb 2022 02:27:21 +0000 (GMT) Received: from sig-9-65-84-116.ibm.com (unknown [9.65.84.116]) by b01ledav006.gho.pok.ibm.com (Postfix) with ESMTP; Sun, 6 Feb 2022 02:27:21 +0000 (GMT) Message-ID: <9ffeb5567b2836ee8326d530310e4eb4f6f9f1f2.camel@linux.ibm.com> Subject: [PATCH v2 1/4] powerpc/pseries/vas: Define global hv_cop_caps struct From: Haren Myneni To: mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, nathanl@linux.ibm.com Date: Sat, 05 Feb 2022 18:27:19 -0800 In-Reply-To: References: User-Agent: Evolution 3.36.5 (3.36.5-2.fc32) MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: AWQHLOupjLqLRS-Gx3ZD5KLc-pXTLtIo X-Proofpoint-GUID: 1VhWo7qyIYQXBxddzomyZ_nBPRwvCbwE X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.816,Hydra:6.0.425,FMLib:17.11.62.513 definitions=2022-02-05_15,2022-02-03_01,2021-12-02_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 phishscore=0 impostorscore=0 bulkscore=0 adultscore=0 priorityscore=1501 suspectscore=0 mlxscore=0 mlxlogscore=999 spamscore=0 malwarescore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2202060011 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" The coprocessor capabilities struct is used to get default and QoS capabilities from the hypervisor during init, DLPAR event and migration. So instead of allocating this struct for each event, define global struct and reuse it, especially eliminate memory allocation failure during migration. Also disable copy/paste feature flag if any capabilities HCALL is failed. Signed-off-by: Haren Myneni Acked-by: Nathan Lynch --- arch/powerpc/platforms/pseries/vas.c | 49 ++++++++++++---------------- 1 file changed, 21 insertions(+), 28 deletions(-) diff --git a/arch/powerpc/platforms/pseries/vas.c b/arch/powerpc/platforms/pseries/vas.c index 3400f4fc6609..933506b85dc6 100644 --- a/arch/powerpc/platforms/pseries/vas.c +++ b/arch/powerpc/platforms/pseries/vas.c @@ -26,6 +26,7 @@ static struct vas_all_caps caps_all; static bool copypaste_feat; +static struct hv_vas_cop_feat_caps hv_cop_caps; static struct vas_caps vascaps[VAS_MAX_FEAT_TYPE]; static DEFINE_MUTEX(vas_pseries_mutex); @@ -724,7 +725,6 @@ static int reconfig_close_windows(struct vas_caps *vcap, int excess_creds) */ int vas_reconfig_capabilties(u8 type) { - struct hv_vas_cop_feat_caps *hv_caps; struct vas_cop_feat_caps *caps; int lpar_creds, new_creds; struct vas_caps *vcaps; @@ -738,17 +738,13 @@ int vas_reconfig_capabilties(u8 type) vcaps = &vascaps[type]; caps = &vcaps->caps; - hv_caps = kmalloc(sizeof(*hv_caps), GFP_KERNEL); - if (!hv_caps) - return -ENOMEM; - mutex_lock(&vas_pseries_mutex); rc = h_query_vas_capabilities(H_QUERY_VAS_CAPABILITIES, vcaps->feat, - (u64)virt_to_phys(hv_caps)); + (u64)virt_to_phys(&hv_cop_caps)); if (rc) goto out; - new_creds = be16_to_cpu(hv_caps->target_lpar_creds); + new_creds = be16_to_cpu(hv_cop_caps.target_lpar_creds); lpar_creds = atomic_read(&caps->target_creds); @@ -778,7 +774,6 @@ int vas_reconfig_capabilties(u8 type) out: mutex_unlock(&vas_pseries_mutex); - kfree(hv_caps); return rc; } @@ -821,9 +816,8 @@ static struct notifier_block pseries_vas_nb = { static int __init pseries_vas_init(void) { - struct hv_vas_cop_feat_caps *hv_cop_caps; struct hv_vas_all_caps *hv_caps; - int rc; + int rc = 0; /* * Linux supports user space COPY/PASTE only with Radix @@ -849,39 +843,38 @@ static int __init pseries_vas_init(void) sysfs_pseries_vas_init(&caps_all); - hv_cop_caps = kmalloc(sizeof(*hv_cop_caps), GFP_KERNEL); - if (!hv_cop_caps) { - rc = -ENOMEM; - goto out; - } /* * QOS capabilities available */ if (caps_all.feat_type & VAS_GZIP_QOS_FEAT_BIT) { rc = get_vas_capabilities(VAS_GZIP_QOS_FEAT, - VAS_GZIP_QOS_FEAT_TYPE, hv_cop_caps); + VAS_GZIP_QOS_FEAT_TYPE, &hv_cop_caps); if (rc) - goto out_cop; + goto out; } /* * Default capabilities available */ - if (caps_all.feat_type & VAS_GZIP_DEF_FEAT_BIT) { + if (caps_all.feat_type & VAS_GZIP_DEF_FEAT_BIT) rc = get_vas_capabilities(VAS_GZIP_DEF_FEAT, - VAS_GZIP_DEF_FEAT_TYPE, hv_cop_caps); - if (rc) - goto out_cop; - } + VAS_GZIP_DEF_FEAT_TYPE, &hv_cop_caps); - /* Processors can be added/removed only on LPAR */ - if (copypaste_feat && firmware_has_feature(FW_FEATURE_LPAR)) - of_reconfig_notifier_register(&pseries_vas_nb); + if (!rc && copypaste_feat) { + /* Processors can be added/removed only on LPAR */ + if (firmware_has_feature(FW_FEATURE_LPAR)) + of_reconfig_notifier_register(&pseries_vas_nb); - pr_info("GZIP feature is available\n"); + pr_info("GZIP feature is available\n"); + } else { + /* + * Should not happen, but only when get default + * capabilities HCALL failed. So disable copy paste + * feature. + */ + copypaste_feat = false; + } -out_cop: - kfree(hv_cop_caps); out: kfree(hv_caps); return rc; From patchwork Sun Feb 6 02:28:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haren Myneni X-Patchwork-Id: 1588891 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=FpMlT9rV; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4JrtVq54fWz9s1l for ; Sun, 6 Feb 2022 13:28:35 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4JrtVq2Wsjz3bb0 for ; Sun, 6 Feb 2022 13:28:35 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=FpMlT9rV; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0b-001b2d01.pphosted.com; envelope-from=haren@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=FpMlT9rV; dkim-atps=neutral Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4JrtVX0Db4z30NP for ; Sun, 6 Feb 2022 13:28:19 +1100 (AEDT) Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 215NY6XF031567; Sun, 6 Feb 2022 02:28:14 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=message-id : subject : from : to : cc : date : in-reply-to : references : content-type : mime-version : content-transfer-encoding; s=pp1; bh=FQWDeD2C90rprqi/rSSZWAWe9jkEm9K4a5qoCv8GM3Y=; b=FpMlT9rVC3+WW5w/UZ2dMbxlVICrRVdJLH4JYS3yew2TYOa9aKMBDlqSh2jnbHnz79hn oU5YXmpf0vYEVbm1OFcsSwW6kQ9zpRgzitGVWRMXJT+2iALwU/lcP0aWTZJCBjjr7C1C XlQQelQZHs+b7m8KxLYvCTwcxrmxc4A5anCbasoHk8KyXgRxv05YAIoxlsSa+1fnRwXn u1RVEnJ8xNicyn3mE8LDK2rD79jsba8UbUKUh+yd8UCX/TAmDcoHGqITe0XIs71bR3/g lyk2j0kbW05GsRh3oV+b9ai0T9MJuC0+aPzWZGx68/jt1KvVkiLrb9mHN56l8Cmv/Web DQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 3e23189qaj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 06 Feb 2022 02:28:14 +0000 Received: from m0098421.ppops.net (m0098421.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 2162SDMZ010996; Sun, 6 Feb 2022 02:28:13 GMT Received: from ppma01dal.us.ibm.com (83.d6.3fa9.ip4.static.sl-reverse.com [169.63.214.131]) by mx0a-001b2d01.pphosted.com with ESMTP id 3e23189qad-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 06 Feb 2022 02:28:13 +0000 Received: from pps.filterd (ppma01dal.us.ibm.com [127.0.0.1]) by ppma01dal.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 2162RFt7003986; Sun, 6 Feb 2022 02:28:12 GMT Received: from b03cxnp07029.gho.boulder.ibm.com (b03cxnp07029.gho.boulder.ibm.com [9.17.130.16]) by ppma01dal.us.ibm.com with ESMTP id 3e1gvabxm3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 06 Feb 2022 02:28:12 +0000 Received: from b03ledav001.gho.boulder.ibm.com (b03ledav001.gho.boulder.ibm.com [9.17.130.232]) by b03cxnp07029.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2162S9US34931110 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sun, 6 Feb 2022 02:28:09 GMT Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1150E6E052; Sun, 6 Feb 2022 02:28:09 +0000 (GMT) Received: from b03ledav001.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 14EC36E050; Sun, 6 Feb 2022 02:28:08 +0000 (GMT) Received: from sig-9-65-84-116.ibm.com (unknown [9.65.84.116]) by b03ledav001.gho.boulder.ibm.com (Postfix) with ESMTP; Sun, 6 Feb 2022 02:28:07 +0000 (GMT) Message-ID: <1bf38930930130fa2cbde27ac22a234184995a93.camel@linux.ibm.com> Subject: [PATCH v2 2/4] powerpc/pseries/vas: Modify reconfig open/close functions for migration From: Haren Myneni To: mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, nathanl@linux.ibm.com Date: Sat, 05 Feb 2022 18:28:06 -0800 In-Reply-To: References: User-Agent: Evolution 3.36.5 (3.36.5-2.fc32) MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: epn9kHtfRqVg0pl1aNU47m7iQT0H-Q7X X-Proofpoint-GUID: XOg0EnbCLy5WD-b4TTGYo7eVFipmiXv_ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.816,Hydra:6.0.425,FMLib:17.11.62.513 definitions=2022-02-05_15,2022-02-03_01,2021-12-02_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 lowpriorityscore=0 malwarescore=0 impostorscore=0 adultscore=0 mlxlogscore=999 suspectscore=0 spamscore=0 bulkscore=0 mlxscore=0 phishscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2202060011 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" VAS is a hardware engine stays on the chip. So when the partition migrates, all VAS windows on the source system have to be closed and reopen them on the destination after migration. This patch make changes to the current reconfig_open/close_windows functions to support migration: - Set VAS_WIN_MIGRATE_CLOSE to the window status when closes and reopen windows with the same status during resume. - Continue to close all windows even if deallocate HCALL failed (should not happen) since no way to stop migration with the current LPM implementation. - If the DLPAR CPU event happens while migration is in progress, set VAS_WIN_NO_CRED_CLOSE to the window status. Close window happens with the first event (migration or DLPAR) and Reopen window happens only with the last event (migration or DLPAR). Signed-off-by: Haren Myneni --- arch/powerpc/include/asm/vas.h | 2 + arch/powerpc/platforms/pseries/vas.c | 88 ++++++++++++++++++++++------ 2 files changed, 73 insertions(+), 17 deletions(-) diff --git a/arch/powerpc/include/asm/vas.h b/arch/powerpc/include/asm/vas.h index ddc05a8fc2e3..f21e76f47175 100644 --- a/arch/powerpc/include/asm/vas.h +++ b/arch/powerpc/include/asm/vas.h @@ -42,6 +42,8 @@ /* Linux status bits */ #define VAS_WIN_NO_CRED_CLOSE 0x00000004 /* Window is closed due to */ /* lost credit */ +#define VAS_WIN_MIGRATE_CLOSE 0x00000008 /* Window is closed due to */ + /* migration */ /* * Get/Set bit fields */ diff --git a/arch/powerpc/platforms/pseries/vas.c b/arch/powerpc/platforms/pseries/vas.c index 933506b85dc6..af495b6f64cc 100644 --- a/arch/powerpc/platforms/pseries/vas.c +++ b/arch/powerpc/platforms/pseries/vas.c @@ -457,11 +457,12 @@ static int vas_deallocate_window(struct vas_window *vwin) mutex_lock(&vas_pseries_mutex); /* * VAS window is already closed in the hypervisor when - * lost the credit. So just remove the entry from - * the list, remove task references and free vas_window + * lost the credit or with migration. So just remove the entry + * from the list, remove task references and free vas_window * struct. */ - if (win->vas_win.status & VAS_WIN_NO_CRED_CLOSE) { + if (!(win->vas_win.status & VAS_WIN_NO_CRED_CLOSE) && + !(win->vas_win.status & VAS_WIN_MIGRATE_CLOSE)) { rc = deallocate_free_window(win); if (rc) { mutex_unlock(&vas_pseries_mutex); @@ -578,12 +579,14 @@ static int __init get_vas_capabilities(u8 feat, enum vas_cop_feat_type type, * by setting the remapping to new paste address if the window is * active. */ -static int reconfig_open_windows(struct vas_caps *vcaps, int creds) +static int reconfig_open_windows(struct vas_caps *vcaps, int creds, + bool migrate) { long domain[PLPAR_HCALL9_BUFSIZE] = {VAS_DEFAULT_DOMAIN_ID}; struct vas_cop_feat_caps *caps = &vcaps->caps; struct pseries_vas_window *win = NULL, *tmp; int rc, mv_ents = 0; + int flag; /* * Nothing to do if there are no closed windows. @@ -602,8 +605,10 @@ static int reconfig_open_windows(struct vas_caps *vcaps, int creds) * (dedicated). If 1 core is added, this LPAR can have 20 more * credits. It means the kernel can reopen 20 windows. So move * 20 entries in the VAS windows lost and reopen next 20 windows. + * For partition migration, reopen all windows that are closed + * during resume. */ - if (vcaps->close_wins > creds) + if ((vcaps->close_wins > creds) && !migrate) mv_ents = vcaps->close_wins - creds; list_for_each_entry_safe(win, tmp, &vcaps->list, win_list) { @@ -613,12 +618,35 @@ static int reconfig_open_windows(struct vas_caps *vcaps, int creds) mv_ents--; } + /* + * Open windows if they are closed only with migration or + * DLPAR (lost credit) before. + */ + if (migrate) + flag = VAS_WIN_MIGRATE_CLOSE; + else + flag = VAS_WIN_NO_CRED_CLOSE; + list_for_each_entry_safe_from(win, tmp, &vcaps->list, win_list) { + /* + * This window is closed with DLPAR and migration events. + * So reopen the window with the last event. + * The user space is not suspended with the current + * migration notifier. So the user space can issue DLPAR + * CPU hotplug while migration in progress. In this case + * this window will be opened with the last event. + */ + if ((win->vas_win.status & VAS_WIN_NO_CRED_CLOSE) && + (win->vas_win.status & VAS_WIN_MIGRATE_CLOSE)) { + win->vas_win.status &= ~flag; + continue; + } + /* * Nothing to do on this window if it is not closed - * with VAS_WIN_NO_CRED_CLOSE + * with this flag */ - if (!(win->vas_win.status & VAS_WIN_NO_CRED_CLOSE)) + if (!(win->vas_win.status & flag)) continue; rc = allocate_setup_window(win, (u64 *)&domain[0], @@ -634,7 +662,7 @@ static int reconfig_open_windows(struct vas_caps *vcaps, int creds) /* * Set window status to active */ - win->vas_win.status &= ~VAS_WIN_NO_CRED_CLOSE; + win->vas_win.status &= ~flag; mutex_unlock(&win->vas_win.task_ref.mmap_mutex); win->win_type = caps->win_type; if (!--vcaps->close_wins) @@ -661,20 +689,32 @@ static int reconfig_open_windows(struct vas_caps *vcaps, int creds) * the user space to fall back to SW compression and manage with the * existing windows. */ -static int reconfig_close_windows(struct vas_caps *vcap, int excess_creds) +static int reconfig_close_windows(struct vas_caps *vcap, int excess_creds, + bool migrate) { struct pseries_vas_window *win, *tmp; struct vas_user_win_ref *task_ref; struct vm_area_struct *vma; - int rc = 0; + int rc = 0, flag; + + if (migrate) + flag = VAS_WIN_MIGRATE_CLOSE; + else + flag = VAS_WIN_NO_CRED_CLOSE; list_for_each_entry_safe(win, tmp, &vcap->list, win_list) { /* * This window is already closed due to lost credit - * before. Go for next window. + * or for migration before. Go for next window. + * For migration, nothing to do since this window + * closed for DLPAR and will be reopened even on + * the destination system with other DLPAR operation. */ - if (win->vas_win.status & VAS_WIN_NO_CRED_CLOSE) + if ((win->vas_win.status & VAS_WIN_MIGRATE_CLOSE) || + (win->vas_win.status & VAS_WIN_NO_CRED_CLOSE)) { + win->vas_win.status |= flag; continue; + } task_ref = &win->vas_win.task_ref; mutex_lock(&task_ref->mmap_mutex); @@ -683,7 +723,7 @@ static int reconfig_close_windows(struct vas_caps *vcap, int excess_creds) * Number of available credits are reduced, So select * and close windows. */ - win->vas_win.status |= VAS_WIN_NO_CRED_CLOSE; + win->vas_win.status |= flag; mmap_write_lock(task_ref->mm); /* @@ -706,12 +746,24 @@ static int reconfig_close_windows(struct vas_caps *vcap, int excess_creds) * later when the process issued with close(FD). */ rc = deallocate_free_window(win); - if (rc) + /* + * This failure is from the hypervisor. + * No way to stop migration for these failures. + * So ignore error and continue closing other windows. + */ + if (rc && !migrate) return rc; vcap->close_wins++; - if (!--excess_creds) + /* + * For migration, do not depend on lpar_creds in case if + * mismatch with the hypervisor value (should not happen). + * So close all active windows in the list and will be + * reopened windows based on the new lpar_creds on the + * destination system during resume. + */ + if (!migrate && !--excess_creds) break; } @@ -761,7 +813,8 @@ int vas_reconfig_capabilties(u8 type) * target, reopen windows if they are closed due to * the previous DLPAR (core removal). */ - rc = reconfig_open_windows(vcaps, new_creds - lpar_creds); + rc = reconfig_open_windows(vcaps, new_creds - lpar_creds, + false); } else { /* * # active windows is more than new LPAR available @@ -769,7 +822,8 @@ int vas_reconfig_capabilties(u8 type) */ active_wins = vcaps->num_wins - vcaps->close_wins; if (active_wins > new_creds) - rc = reconfig_close_windows(vcaps, active_wins - new_creds); + rc = reconfig_close_windows(vcaps, active_wins - new_creds, + false); } out: From patchwork Sun Feb 6 02:28:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haren Myneni X-Patchwork-Id: 1588892 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=NaL/mmW8; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4JrtWg1Hsgz9s1l for ; Sun, 6 Feb 2022 13:29:19 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4JrtWf6KJCz3bT0 for ; Sun, 6 Feb 2022 13:29:18 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=NaL/mmW8; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=haren@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=NaL/mmW8; dkim-atps=neutral Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4JrtWP3k61z3bPM for ; Sun, 6 Feb 2022 13:29:05 +1100 (AEDT) Received: from pps.filterd (m0098413.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 215NQFdH006390; Sun, 6 Feb 2022 02:29:00 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=message-id : subject : from : to : cc : date : in-reply-to : references : content-type : mime-version : content-transfer-encoding; s=pp1; bh=T/NhOdKnu9b1spBjrkx6DLYQ8dRW7ZxvFiEO2eIlxdo=; b=NaL/mmW8MKkk1Br5PdiNh9rfl+RLijvL9FiFKPdgRa8Dz01vpamM6wW/8ipYzD/0LTxP cWSGoEl4yxxphu5Qwac6ohdXrcMkZsMbM8PVG40Od8TsnaMnm9kubMvMBiy9X7UCITdd CE7i8nOVQCcaI6NuU1PDgcgxYmOLD9YGGpYkeVUmM5bZnSjnTRrbG7j07rBz3IBk5c5O fR70PsAWB2kZpiVgF7+605qs4plGsBiAl8esQLvhAsXh4xjMRos2s02w1IojtXMgipfF WWPmVTY0hOVGO/+2JCgx/iGpXnH+2I7Gvn0LmpibC86L2DniQpyC8DQclfqFnJHs7ZK3 uw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 3e22wghr9r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 06 Feb 2022 02:29:00 +0000 Received: from m0098413.ppops.net (m0098413.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 2162SxfM023644; Sun, 6 Feb 2022 02:29:00 GMT Received: from ppma04wdc.us.ibm.com (1a.90.2fa9.ip4.static.sl-reverse.com [169.47.144.26]) by mx0b-001b2d01.pphosted.com with ESMTP id 3e22wghr9n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 06 Feb 2022 02:28:59 +0000 Received: from pps.filterd (ppma04wdc.us.ibm.com [127.0.0.1]) by ppma04wdc.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 2162RMQV003444; Sun, 6 Feb 2022 02:28:59 GMT Received: from b03cxnp08026.gho.boulder.ibm.com (b03cxnp08026.gho.boulder.ibm.com [9.17.130.18]) by ppma04wdc.us.ibm.com with ESMTP id 3e1gv9hkte-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 06 Feb 2022 02:28:59 +0000 Received: from b03ledav006.gho.boulder.ibm.com (b03ledav006.gho.boulder.ibm.com [9.17.130.237]) by b03cxnp08026.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2162SsIm31654364 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sun, 6 Feb 2022 02:28:54 GMT Received: from b03ledav006.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 67790C6057; Sun, 6 Feb 2022 02:28:54 +0000 (GMT) Received: from b03ledav006.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 63B8CC6055; Sun, 6 Feb 2022 02:28:53 +0000 (GMT) Received: from sig-9-65-84-116.ibm.com (unknown [9.65.84.116]) by b03ledav006.gho.boulder.ibm.com (Postfix) with ESMTP; Sun, 6 Feb 2022 02:28:53 +0000 (GMT) Message-ID: Subject: [PATCH v2 3/4] powerpc/pseries/vas: Add VAS migration handler From: Haren Myneni To: mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, nathanl@linux.ibm.com Date: Sat, 05 Feb 2022 18:28:51 -0800 In-Reply-To: References: User-Agent: Evolution 3.36.5 (3.36.5-2.fc32) MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: 0VS6MV7zCiEdNGNldWyCjwHTBC85H9L0 X-Proofpoint-ORIG-GUID: nlDzpSATexvINpfemC2hkmbJe3ZnzJe6 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.816,Hydra:6.0.425,FMLib:17.11.62.513 definitions=2022-02-05_15,2022-02-03_01,2021-12-02_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 impostorscore=0 mlxlogscore=999 clxscore=1015 priorityscore=1501 bulkscore=0 spamscore=0 adultscore=0 mlxscore=0 phishscore=0 suspectscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2202060011 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" Since the VAS windows belong to the VAS hardware resource, the hypervisor expects the partition to close them on source partition and reopen them after the partition migrated on the destination machine. This handler is called before pseries_suspend() to close these windows and again invoked after migration. All active windows for both default and QoS types will be closed and mark them in-active and reopened after migration with this handler. During the migration, the user space receives paste instruction failure if it issues copy/paste on these in-active windows. Signed-off-by: Haren Myneni --- arch/powerpc/platforms/pseries/mobility.c | 5 ++ arch/powerpc/platforms/pseries/vas.c | 86 +++++++++++++++++++++++ arch/powerpc/platforms/pseries/vas.h | 6 ++ 3 files changed, 97 insertions(+) diff --git a/arch/powerpc/platforms/pseries/mobility.c b/arch/powerpc/platforms/pseries/mobility.c index 85033f392c78..70004243e25e 100644 --- a/arch/powerpc/platforms/pseries/mobility.c +++ b/arch/powerpc/platforms/pseries/mobility.c @@ -26,6 +26,7 @@ #include #include #include "pseries.h" +#include "vas.h" /* vas_migration_handler() */ #include "../../kernel/cacheinfo.h" static struct kobject *mobility_kobj; @@ -669,12 +670,16 @@ static int pseries_migrate_partition(u64 handle) if (ret) return ret; + vas_migration_handler(VAS_SUSPEND); + ret = pseries_suspend(handle); if (ret == 0) post_mobility_fixup(); else pseries_cancel_migration(handle, ret); + vas_migration_handler(VAS_RESUME); + return ret; } diff --git a/arch/powerpc/platforms/pseries/vas.c b/arch/powerpc/platforms/pseries/vas.c index af495b6f64cc..345d469c3560 100644 --- a/arch/powerpc/platforms/pseries/vas.c +++ b/arch/powerpc/platforms/pseries/vas.c @@ -868,6 +868,92 @@ static struct notifier_block pseries_vas_nb = { .notifier_call = pseries_vas_notifier, }; +/* + * For LPM, all windows have to be closed on the source partition + * before migration and reopen them on the destination partition + * after migration. So closing windows during suspend and + * reopen them during resume. + */ +int vas_migration_handler(int action) +{ + struct vas_cop_feat_caps *caps; + int lpar_creds, new_creds = 0; + struct vas_caps *vcaps; + int i, rc = 0; + + /* + * NX-GZIP is not enabled. Nothing to do for migration. + */ + if (!copypaste_feat) + return rc; + + mutex_lock(&vas_pseries_mutex); + + for (i = 0; i < VAS_MAX_FEAT_TYPE; i++) { + vcaps = &vascaps[i]; + caps = &vcaps->caps; + lpar_creds = atomic_read(&caps->target_creds); + + rc = h_query_vas_capabilities(H_QUERY_VAS_CAPABILITIES, + vcaps->feat, + (u64)virt_to_phys(&hv_cop_caps)); + if (!rc) { + new_creds = be16_to_cpu(hv_cop_caps.target_lpar_creds); + /* + * Should not happen. But incase print messages, close + * all windows in the list during suspend and reopen + * windows based on new lpar_creds on the destination + * system. + */ + if (lpar_creds != new_creds) { + pr_err("state(%d): lpar creds: %d HV lpar creds: %d\n", + action, lpar_creds, new_creds); + pr_err("Used creds: %d, Active creds: %d\n", + atomic_read(&caps->used_creds), + vcaps->num_wins - vcaps->close_wins); + } + } else { + pr_err("state(%d): Get VAS capabilities failed with %d\n", + action, rc); + /* + * We can not stop migration with the current lpm + * implementation. So continue closing all windows in + * the list (during suspend) and return without + * opening windows (during resume) if VAS capabilities + * HCALL failed. + */ + if (action == VAS_RESUME) + goto out; + } + + switch (action) { + case VAS_SUSPEND: + rc = reconfig_close_windows(vcaps, vcaps->num_wins, + true); + break; + case VAS_RESUME: + atomic_set(&caps->target_creds, new_creds); + rc = reconfig_open_windows(vcaps, new_creds, true); + break; + default: + /* should not happen */ + pr_err("Invalid migration action %d\n", action); + rc = -EINVAL; + goto out; + } + + /* + * Ignore errors during suspend and return for resume. + */ + if (rc && (action == VAS_RESUME)) + goto out; + } + +out: + mutex_unlock(&vas_pseries_mutex); + return rc; +} + static int __init pseries_vas_init(void) { struct hv_vas_all_caps *hv_caps; diff --git a/arch/powerpc/platforms/pseries/vas.h b/arch/powerpc/platforms/pseries/vas.h index 4cf1d0ef66a5..36dd140af01b 100644 --- a/arch/powerpc/platforms/pseries/vas.h +++ b/arch/powerpc/platforms/pseries/vas.h @@ -27,6 +27,11 @@ #define VAS_GZIP_QOS_CAPABILITIES 0x56516F73477A6970 #define VAS_GZIP_DEFAULT_CAPABILITIES 0x56446566477A6970 +enum vas_migrate_action { + VAS_SUSPEND, + VAS_RESUME, +}; + /* * Co-processor feature - GZIP QoS windows or GZIP default windows */ @@ -127,4 +132,5 @@ struct pseries_vas_window { int sysfs_add_vas_caps(struct vas_cop_feat_caps *caps); int vas_reconfig_capabilties(u8 type); int __init sysfs_pseries_vas_init(struct vas_all_caps *vas_caps); +int vas_migration_handler(int action); #endif /* _VAS_H */ From patchwork Sun Feb 6 02:29:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haren Myneni X-Patchwork-Id: 1588893 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=rTCsXp8+; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ozlabs.org (client-ip=112.213.38.117; helo=lists.ozlabs.org; envelope-from=linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org; receiver=) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4JrtXp5NXmz9s1l for ; Sun, 6 Feb 2022 13:30:18 +1100 (AEDT) Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4JrtXp3QfRz3cSf for ; Sun, 6 Feb 2022 13:30:18 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=rTCsXp8+; dkim-atps=neutral X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=linux.ibm.com (client-ip=148.163.158.5; helo=mx0a-001b2d01.pphosted.com; envelope-from=haren@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=ibm.com header.i=@ibm.com header.a=rsa-sha256 header.s=pp1 header.b=rTCsXp8+; dkim-atps=neutral Received: from mx0a-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4JrtXW3Brzz3bPM for ; Sun, 6 Feb 2022 13:30:03 +1100 (AEDT) Received: from pps.filterd (m0098419.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 215N53Gv020006; Sun, 6 Feb 2022 02:29:58 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=message-id : subject : from : to : cc : date : in-reply-to : references : content-type : mime-version : content-transfer-encoding; s=pp1; bh=62dhh2B6eFHn2OqA/jsCmk+qp0I1FxqUCtclI1+MdAs=; b=rTCsXp8+Yo5u3sqjin3MoOXPpjsiS0bj0aSnYg9/XhlnzQVTgqHGpxa/Jv/uVUSlK9kb Lijuvxftkg/TkfgwgzUtLXdok9QePpY3fHOEvSShiRa/88QJY7mjoKJbSqhThR5SYkVb JZqCq21DiS7HH6WQFza5OSrVND3OK70QSvfyeRu+S9lAwqeyRylVvVa4eIGCOo0NbQbx XBj4X5jG8ZTGWkSITgYDSSBz6SVzq2pm0LEQESBivGcWt7qwZql9n9rIywIxFX1LI8q4 seYNk+IZD8aG7Vzf/rRk6WIzQbSch/+vfL1fLePOXnS1OCxTwy9YwZxKkHkA33+kMFY2 Ug== Received: from pps.reinject (localhost [127.0.0.1]) by mx0b-001b2d01.pphosted.com with ESMTP id 3e22kp1ywa-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 06 Feb 2022 02:29:58 +0000 Received: from m0098419.ppops.net (m0098419.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 2162Tl9w006580; Sun, 6 Feb 2022 02:29:58 GMT Received: from ppma02wdc.us.ibm.com (aa.5b.37a9.ip4.static.sl-reverse.com [169.55.91.170]) by mx0b-001b2d01.pphosted.com with ESMTP id 3e22kp1yw5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 06 Feb 2022 02:29:57 +0000 Received: from pps.filterd (ppma02wdc.us.ibm.com [127.0.0.1]) by ppma02wdc.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 2162Sldq020988; Sun, 6 Feb 2022 02:29:57 GMT Received: from b01cxnp22033.gho.pok.ibm.com (b01cxnp22033.gho.pok.ibm.com [9.57.198.23]) by ppma02wdc.us.ibm.com with ESMTP id 3e1gv99knw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 06 Feb 2022 02:29:57 +0000 Received: from b01ledav005.gho.pok.ibm.com (b01ledav005.gho.pok.ibm.com [9.57.199.110]) by b01cxnp22033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2162TtZf32965114 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sun, 6 Feb 2022 02:29:55 GMT Received: from b01ledav005.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 86694AE064; Sun, 6 Feb 2022 02:29:55 +0000 (GMT) Received: from b01ledav005.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D45B4AE05C; Sun, 6 Feb 2022 02:29:54 +0000 (GMT) Received: from sig-9-65-84-116.ibm.com (unknown [9.65.84.116]) by b01ledav005.gho.pok.ibm.com (Postfix) with ESMTP; Sun, 6 Feb 2022 02:29:54 +0000 (GMT) Message-ID: <50abb909d5d9dff020bad754e3437ec23d90bc56.camel@linux.ibm.com> Subject: [PATCH v2 4/4] powerpc/pseries/vas: Disable window open during migration From: Haren Myneni To: mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, nathanl@linux.ibm.com Date: Sat, 05 Feb 2022 18:29:52 -0800 In-Reply-To: References: User-Agent: Evolution 3.36.5 (3.36.5-2.fc32) MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: EP3_hcQ0nv5pD7WXIUuR1KyccaZXEDyS X-Proofpoint-ORIG-GUID: -JtYTJL1lt3Z7ywks3NjfOFGx22rS2Um X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.816,Hydra:6.0.425,FMLib:17.11.62.513 definitions=2022-02-05_15,2022-02-03_01,2021-12-02_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 priorityscore=1501 bulkscore=0 spamscore=0 impostorscore=0 suspectscore=0 mlxlogscore=817 lowpriorityscore=0 adultscore=0 malwarescore=0 mlxscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2202060011 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linuxppc-dev-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" The current partition migration implementation does not freeze the user space and the user space can continue open VAS windows. So when migration_in_progress flag is enabled, VAS open window API returns -EBUSY. Signed-off-by: Haren Myneni --- arch/powerpc/platforms/pseries/vas.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/arch/powerpc/platforms/pseries/vas.c b/arch/powerpc/platforms/pseries/vas.c index 345d469c3560..ffcf18cfb4e3 100644 --- a/arch/powerpc/platforms/pseries/vas.c +++ b/arch/powerpc/platforms/pseries/vas.c @@ -30,6 +30,7 @@ static struct hv_vas_cop_feat_caps hv_cop_caps; static struct vas_caps vascaps[VAS_MAX_FEAT_TYPE]; static DEFINE_MUTEX(vas_pseries_mutex); +static bool migration_in_progress; static long hcall_return_busy_check(long rc) { @@ -356,8 +357,11 @@ static struct vas_window *vas_allocate_window(int vas_id, u64 flags, * same fault IRQ is not freed by the OS before. */ mutex_lock(&vas_pseries_mutex); - rc = allocate_setup_window(txwin, (u64 *)&domain[0], - cop_feat_caps->win_type); + if (migration_in_progress) + rc = -EBUSY; + else + rc = allocate_setup_window(txwin, (u64 *)&domain[0], + cop_feat_caps->win_type); mutex_unlock(&vas_pseries_mutex); if (rc) goto out; @@ -889,6 +893,11 @@ int vas_migration_handler(int action) mutex_lock(&vas_pseries_mutex); + if (action == VAS_SUSPEND) + migration_in_progress = true; + else + migration_in_progress = false; + for (i = 0; i < VAS_MAX_FEAT_TYPE; i++) { vcaps = &vascaps[i]; caps = &vcaps->caps;