From patchwork Fri Jul 27 13:51:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Dufour X-Patchwork-Id: 950184 Return-Path: X-Original-To: patchwork-incoming@ozlabs.org Delivered-To: patchwork-incoming@ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41cVnb24Tpz9rxx for ; Fri, 27 Jul 2018 23:54:27 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.vnet.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 41cVnb0WGbzF0wy for ; Fri, 27 Jul 2018 23:54:27 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.vnet.ibm.com X-Original-To: linuxppc-dev@lists.ozlabs.org Delivered-To: linuxppc-dev@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=none (mailfrom) smtp.mailfrom=linux.vnet.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=ldufour@linux.vnet.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.vnet.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 41cVkS6f9yzDqkJ for ; Fri, 27 Jul 2018 23:51:44 +1000 (AEST) Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w6RDmind048743 for ; Fri, 27 Jul 2018 09:51:42 -0400 Received: from e06smtp04.uk.ibm.com (e06smtp04.uk.ibm.com [195.75.94.100]) by mx0a-001b2d01.pphosted.com with ESMTP id 2kg3633wv4-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 27 Jul 2018 09:51:41 -0400 Received: from localhost by e06smtp04.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 27 Jul 2018 14:51:39 +0100 Received: from b06cxnps3074.portsmouth.uk.ibm.com (9.149.109.194) by e06smtp04.uk.ibm.com (192.168.101.134) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Fri, 27 Jul 2018 14:51:35 +0100 Received: from d06av21.portsmouth.uk.ibm.com (d06av21.portsmouth.uk.ibm.com [9.149.105.232]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w6RDpYZp41484478 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Fri, 27 Jul 2018 13:51:34 GMT Received: from d06av21.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4077652057; Fri, 27 Jul 2018 16:51:47 +0100 (BST) Received: from nimbus.lab.toulouse-stg.fr.ibm.com (unknown [9.145.62.135]) by d06av21.portsmouth.uk.ibm.com (Postfix) with ESMTP id B3D1A5204E; Fri, 27 Jul 2018 16:51:46 +0100 (BST) From: Laurent Dufour To: linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org Subject: [resend] [PATCH 0/3] powerpc/pseries: use H_BLOCK_REMOVE Date: Fri, 27 Jul 2018 15:51:30 +0200 X-Mailer: git-send-email 2.7.4 X-TM-AS-GCONF: 00 x-cbid: 18072713-0016-0000-0000-000001ECE165 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18072713-0017-0000-0000-00003241EEA0 Message-Id: <1532699493-10883-1-git-send-email-ldufour@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-07-27_06:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=396 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1806210000 definitions=main-1807270142 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aneesh.kumar@linux.ibm.com, npiggin@gmail.com, paulus@samba.org Errors-To: linuxppc-dev-bounces+patchwork-incoming=ozlabs.org@lists.ozlabs.org Sender: "Linuxppc-dev" [Resending so everyone is getting the cover letter] On very large system we could see soft lockup fired when a process is exiting watchdog: BUG: soft lockup - CPU#851 stuck for 21s! [forkoff:215523] Modules linked in: pseries_rng rng_core xfs raid10 vmx_crypto btrfs libcrc32c xor zstd_decompress zstd_compress xxhash lzo_compress raid6_pq crc32c_vpmsum lpfc crc_t10dif crct10dif_generic crct10dif_common dm_multipath scsi_dh_rdac scsi_dh_alua autofs4 CPU: 851 PID: 215523 Comm: forkoff Not tainted 4.17.0 #1 NIP: c0000000000b995c LR: c0000000000b8f64 CTR: 000000000000aa18 REGS: c00006b0645b7610 TRAP: 0901 Not tainted (4.17.0) MSR: 800000010280b033 CR: 22042082 XER: 00000000 CFAR: 00000000006cf8f0 SOFTE: 0 GPR00: 0010000000000000 c00006b0645b7890 c000000000f99200 0000000000000000 GPR04: 8e000001a5a4de58 400249cf1bfd5480 8e000001a5a4de50 400249cf1bfd5480 GPR08: 8e000001a5a4de48 400249cf1bfd5480 8e000001a5a4de40 400249cf1bfd5480 GPR12: ffffffffffffffff c00000001e690800 NIP [c0000000000b995c] plpar_hcall9+0x44/0x7c LR [c0000000000b8f64] pSeries_lpar_flush_hash_range+0x324/0x3d0 Call Trace: [c00006b0645b7890] [8e000001a5a4dd20] 0x8e000001a5a4dd20 (unreliable) [c00006b0645b7a00] [c00000000006d5b0] flush_hash_range+0x60/0x110 [c00006b0645b7a50] [c000000000072a2c] __flush_tlb_pending+0x4c/0xd0 [c00006b0645b7a80] [c0000000002eaf44] unmap_page_range+0x984/0xbd0 [c00006b0645b7bc0] [c0000000002eb594] unmap_vmas+0x84/0x100 [c00006b0645b7c10] [c0000000002f8afc] exit_mmap+0xac/0x1f0 [c00006b0645b7cd0] [c0000000000f2638] mmput+0x98/0x1b0 [c00006b0645b7d00] [c0000000000fc9d0] do_exit+0x330/0xc00 [c00006b0645b7dc0] [c0000000000fd384] do_group_exit+0x64/0x100 [c00006b0645b7e00] [c0000000000fd44c] sys_exit_group+0x2c/0x30 [c00006b0645b7e30] [c00000000000b960] system_call+0x58/0x6c Instruction dump: 60000000 f8810028 7ca42b78 7cc53378 7ce63b78 7d074378 7d284b78 7d495378 e9410060 e9610068 e9810070 44000022 <7d806378> e9810028 f88c0000 f8ac0008 This happens when removing the PTE by calling the hypervisor using the H_BULK_REMOVE call. This call is processing up to 4 PTEs but is doing a tlbie for each PTE it is processing. This could lead to long time spent in the hypervisor (sometimes up to 4s) and soft lockup being raised because the scheduler is not called in zap_pte_range(). Since the Power7's time, the hypervisor is providing a new hcall H_BLOCK_REMOVE allowing processing up to 8 PTEs with one call to tlbie. By limiting the amount of tlbie generated, this reduces the time spent invalidating the PTEs. This hcall requires that the pages are "all within the same naturally aligned 8 page virtual address block". With this patch series applied, I couldn't see any soft lockup raised on the victim LPAR I was running the test one. This series is covering both normal pages and huge pages. Laurent Dufour (3): powerpc/pseries/mm: Introducing FW_FEATURE_BLOCK_REMOVE powerpc/pseries/mm: factorize PTE slot computation powerpc/pseries/mm: call H_BLOCK_REMOVE arch/powerpc/include/asm/firmware.h | 3 +- arch/powerpc/include/asm/hvcall.h | 1 + arch/powerpc/platforms/pseries/firmware.c | 1 + arch/powerpc/platforms/pseries/lpar.c | 250 ++++++++++++++++++++++++++---- 4 files changed, 228 insertions(+), 27 deletions(-)