From patchwork Fri Jun 29 11:15:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suzuki K Poulose X-Patchwork-Id: 936810 Return-Path: X-Original-To: incoming-imx@patchwork.ozlabs.org Delivered-To: patchwork-incoming-imx@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.infradead.org (client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org; envelope-from=linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="XyYdyyvl"; dkim-atps=neutral Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41HDjw6RYNz9rvt for ; Fri, 29 Jun 2018 21:21:24 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:To :From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=NEUQiWlZR4tHgVAF5Nc+m0QVzRImqLls0tK9BRZSoJQ=; b=XyYdyyvlzGk2Gi 8i7LuQoN18dX2n8zeBUQ3WrshWporkj1BH7mnIH5dzy8agRR0l0vvjil+RKlkWuxvg0CChhhICF5X fAF4f50XG6iMeRqEgKoLMs0UUs+7SoNV0vw8VlVWwHqHnvvjTHDmPdIV8xz/4pPBee4hjotkf5uXA /9Vw46G7hjKMY2nubma1aNAQ78O0aFuI01/Rxvt8WNt7epTeLSlOHAsuk4NUqb6Hrv8+5vh0bWMQT +sVk97qL9BGBDMKvjttPF3LREhjIhe7lN+VybAePuZCzT7imdbnjIT8gVK8QMlZbjYqlZh730u3Ne X8hH5g6ctNuDmkkWSmVg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fYrSq-0002Dp-LB; Fri, 29 Jun 2018 11:21:20 +0000 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fYrO4-0006Lr-Vh for linux-arm-kernel@lists.infradead.org; Fri, 29 Jun 2018 11:16:29 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C65CD18A; Fri, 29 Jun 2018 04:16:12 -0700 (PDT) Received: from en101.cambridge.arm.com (en101.cambridge.arm.com [10.1.206.73]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 6C14F3F266; Fri, 29 Jun 2018 04:16:10 -0700 (PDT) From: Suzuki K Poulose To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v3 00/20] arm64: Dynamic & 52bit IPA support Date: Fri, 29 Jun 2018 12:15:20 +0100 Message-Id: <1530270944-11351-1-git-send-email-suzuki.poulose@arm.com> X-Mailer: git-send-email 2.7.4 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180629_041625_127813_115EA71E X-CRM114-Status: GOOD ( 27.23 ) X-Spam-Score: -5.0 (-----) X-Spam-Report: SpamAssassin version 3.4.1 on bombadil.infradead.org summary: Content analysis details: (-5.0 points) pts rule name description ---- ---------------------- -------------------------------------------------- -5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/, high trust [217.140.101.70 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cdall@kernel.org, kvm@vger.kernel.org, Suzuki K Poulose , marc.zyngier@arm.com, catalin.marinas@arm.com, punit.agrawal@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, qemu-devel@nongnu.org, eric.auger@redhat.com, julien.grall@arm.com, james.morse@arm.com, kvmarm@lists.cs.columbia.edu Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+incoming-imx=patchwork.ozlabs.org@lists.infradead.org List-Id: linux-imx-kernel.lists.patchwork.ozlabs.org The physical address space size for a VM (IPA size) on arm/arm64 is limited to a static limit of 40bits. This series adds support for using an IPA size specific to a VM, allowing to use a limit supported by the host (based on the host kernel configuration and CPU support). The default and the minimum size is fixed to 40bits. We also add support for handling 52bit IPA addresses added by Arm v8.2 extensions. As mentioned above, the supported IPA size on a host could be different from the system's PARange indicated by the CPUs (e.g, kernel limit on the PA size). So we expose the limit via a new system ioctl request - KVM_ARM_GET_MAX_VM_PHYS_SHIFT - on arm/arm64. This can then be passed on to the KVM_CREATE_VM ioctl, encoded in the "type" field. Bits [7-0] of the type are reserved for the IPA size. This approach allows simpler management of the stage2 page table and guest memory slots. The arm64 page table level helpers are defined based on the page table levels used by the host VA. So, the accessors may not work if the guest uses more number of levels in stage2 than the stage1 of the host. The previous versions (v1 & v2) of this series refactored the stage1 page table accessors to reuse the low-level accessors for an independent stage2 table. However, due to the level folding in the generic code, the types are redefined as well. i.e, if the PUD is folded, the pud_t could be defined as : typedef struct { pgd_t pgd; } pud_t; similarly for pmd_t. So, without stage1 independent page table entry types for stage2, we could be dealing with a different type for level 0-2 entries. This is practically fine on arm/arm64 as the entries have similar format and size and we always use the appropriate accessors to get the raw value (i.e, pud_val/pmd_val etc). But not ideal for a solution upstream. So, this version caps the stage2 page table levels to that of the stage1. This has the following impact on the IPA support for various pagesize/host-va combinations : x-----------------------------------------------------x | host\ipa | 40bit | 42bit | 44bit | 48bit | 52bit | ------------------------------------------------------- | 39bit-4K | y | y | n | n | n/a | ------------------------------------------------------- | 48bit-4K | y | y | y | y | n/a | ------------------------------------------------------- | 36bit-16K | y | n | n | n | n/a | ------------------------------------------------------- | 47bit-16K | y | y | y | y | n/a | ------------------------------------------------------- | 48bit-4K | y | y | y | y | n/a | ------------------------------------------------------- | 42bit-64K | y | y | y | n | n | ------------------------------------------------------- | 48bit-64K | y | y | y | y | y | x-----------------------------------------------------x Or the following list shows what cannot be supported : 39bit-4K host supporting IPA > 43bit (upto 48bit) 36bit-16K host for IPA > 40bit (upto 48bit) 42bit-64K host for IPA > 46bit (upto 52bit) which is not really bad. We can pursue the independent stage2 page table support and lift the restriction once we get there. Given there is a proposal for new generic page table walker [0], it would make sense to make our efforts in sync with it to avoid diverting from a common API. 52bit support is added for VGIC (including ITS emulation) and handling of PAR, HPFAR registers. The series applies on 4.18-rc2. A tree is available here: git://linux-arm.org/linux-skp.git ipa52/v3 Tested with - Modified kvmtool, which can only be used for (patches included in the series for reference / testing): * with virtio-pci upto 44bit PA (Due to 4K page size for virtio-pci legacy implemented by kvmtool) * Upto 48bit PA with virtio-mmio, due to 32bit PFN limitation. - Hacked Qemu (boot loader support for highmem, phys-shift support) * with virtio-pci GIC-v3 ITS & MSI upto 52bit on Foundation model. Also see [1] for Qemu support. [0] https://lkml.org/lkml/2018/4/24/777 [1] https://lists.gnu.org/archive/html/qemu-devel/2018-06/msg05759.html Changes since V2: - Drop "refactoring of host page table helpers" and restrict the IPA size to make sure stage2 doesn't use more page table levels than that of the host. - Load VTCR for TLB operations on behalf of the VM (Pointed-by: James Morse) - Split a couple of patches to make them easier to review. - Fall back to normal (non-concatenated) entry level page table support if possible. - Bump the IOCTL number Changes since V1: - Change the userspace API for configuring VM to encode the IPA size in the VM type. (suggested by Christoffer) - Expose the IPA limit on the host via ioctl on /dev/kvm - Handle 52bit addresses in PAR & HPFAR - Drop patch changing the life time of stage2 PGD - Rename macros for 48-to-52 bit conversion for GIC ITS BASER. (suggested by Christoffer) - Split virtio PFN check patches and address comments. Kristina Martsenko (1): vgic: Add support for 52bit guest physical address Suzuki K Poulose (19): virtio: mmio-v1: Validate queue PFN virtio: pci-legacy: Validate queue pfn arm64: Add a helper for PARange to physical shift conversion kvm: arm64: Clean up VTCR_EL2 initialisation kvm: arm/arm64: Fix stage2_flush_memslot for 4 level page table kvm: arm/arm64: Remove spurious WARN_ON kvm: arm/arm64: Prepare for VM specific stage2 translations kvm: arm/arm64: Abstract stage2 pgd table allocation kvm: arm64: Make stage2 page table layout dynamic kvm: arm64: Dynamic configuration of VTTBR mask kvm: arm64: Helper for computing VTCR_EL2.SL0 kvm: arm64: Add helper for loading the stage2 setting for a VM kvm: arm64: Configure VTCR per VM kvm: arm/arm64: Expose supported physical address limit for VM kvm: arm/arm64: Allow tuning the physical address size for VM kvm: arm64: Switch to per VM IPA limit kvm: arm64: Add support for handling 52bit IPA kvm: arm64: Allow IPA size supported by the system kvm: arm64: Fall back to normal stage2 entry level Documentation/virtual/kvm/api.txt | 15 ++ arch/arm/include/asm/kvm_arm.h | 3 +- arch/arm/include/asm/kvm_mmu.h | 28 +++- arch/arm/include/asm/stage2_pgtable.h | 42 ++--- arch/arm64/include/asm/cpufeature.h | 13 ++ arch/arm64/include/asm/kvm_arm.h | 137 ++++++++++++++--- arch/arm64/include/asm/kvm_asm.h | 2 +- arch/arm64/include/asm/kvm_host.h | 19 ++- arch/arm64/include/asm/kvm_hyp.h | 16 ++ arch/arm64/include/asm/kvm_mmu.h | 92 ++++++++++- arch/arm64/include/asm/stage2_pgtable-nopmd.h | 42 ----- arch/arm64/include/asm/stage2_pgtable-nopud.h | 39 ----- arch/arm64/include/asm/stage2_pgtable.h | 213 +++++++++++++++++++------- arch/arm64/kvm/guest.c | 42 +++++ arch/arm64/kvm/hyp/s2-setup.c | 37 +---- arch/arm64/kvm/hyp/switch.c | 4 +- arch/arm64/kvm/hyp/tlb.c | 4 +- drivers/virtio/virtio_mmio.c | 18 ++- drivers/virtio/virtio_pci_legacy.c | 12 +- include/linux/irqchip/arm-gic-v3.h | 5 + include/uapi/linux/kvm.h | 16 ++ virt/kvm/arm/arm.c | 32 +++- virt/kvm/arm/mmu.c | 124 ++++++++------- virt/kvm/arm/vgic/vgic-its.c | 36 ++--- virt/kvm/arm/vgic/vgic-kvm-device.c | 2 +- virt/kvm/arm/vgic/vgic-mmio-v3.c | 2 - 26 files changed, 663 insertions(+), 332 deletions(-) delete mode 100644 arch/arm64/include/asm/stage2_pgtable-nopmd.h delete mode 100644 arch/arm64/include/asm/stage2_pgtable-nopud.h kvmtool patches : Suzuki K Poulose (4): kvmtool: Allow backends to run checks on the KVM device fd kvmtool: arm64: Add support for guest physical address size kvmtool: arm64: Switch memory layout kvmtool: arm: Add support for creating VM with PA size arm/aarch32/include/kvm/kvm-arch.h | 6 ++++-- arm/aarch64/include/kvm/kvm-arch.h | 15 ++++++++++++--- arm/aarch64/include/kvm/kvm-config-arch.h | 5 ++++- arm/include/arm-common/kvm-arch.h | 17 +++++++++++------ arm/include/arm-common/kvm-config-arch.h | 1 + arm/kvm.c | 24 +++++++++++++++++++++++- include/kvm/kvm.h | 4 ++++ kvm.c | 2 ++ 8 files changed, 61 insertions(+), 13 deletions(-) Reviewed-by: Eric Auger Reviewed-by: Eric Auger Reviewed-by: Eric Auger Reviewed-by: Eric Auger Reviewed-by: Marc Zyngier Acked-by: Marc Zyngier Acked-by: Marc Zyngier Acked-by: Marc Zyngier Reviewed-by: Eric Auger Reviewed-by: Eric Auger Reviewed-by: Eric Auger Reviewed-by: Eric Auger