From patchwork Wed Apr 19 22:17:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Atish Kumar Patra X-Patchwork-Id: 1770974 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.infradead.org (client-ip=2607:7c80:54:3::133; helo=bombadil.infradead.org; envelope-from=kvm-riscv-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org; receiver=) Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; secure) header.d=lists.infradead.org header.i=@lists.infradead.org header.a=rsa-sha256 header.s=bombadil.20210309 header.b=FoodZ/vZ; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=rivosinc-com.20221208.gappssmtp.com header.i=@rivosinc-com.20221208.gappssmtp.com header.a=rsa-sha256 header.s=20221208 header.b=0nUEiKek; dkim-atps=neutral Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:3::133]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4Q1xqc6dKSz1yZk for ; Thu, 20 Apr 2023 09:30:52 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=4KDVuw3g/QNWA7GHXDDmdyglWRyvoPbuZuRv7aD08G8=; b=FoodZ/vZyvQRgV wnM8Q0XukxU9y2kvLxP4UiDyqv148yEUIZr22tCLE8iOp4dvdxUkq3jEPWZZVIfCLHTtz/MSlFj79 L9kCGOZTJgOqTksJ9jjCn3BkD/uR0oUhVk+qmMQ9OWa+n8NdnQenqEDlLkO6yT3y1bTiujdyVm4kK tQlRoE4WTfNAbIo6/ylLPpYG6gvJ+S6OHmnAot0EGcko6TDUaBaSIxiPxYlD/ix+kD193RTkuOzhn ZokQ2b7+Db8LX6861c39Ng/Phu9racxnRl362TL7rkJmX9yaF8MD4G13Jzx2ygecsX9/f69vQryrl 9jGWU8GE0MNrp4ndg3yg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1ppHGF-006egY-0E; Wed, 19 Apr 2023 23:30:51 +0000 Received: from mail-pl1-x630.google.com ([2607:f8b0:4864:20::630]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1ppG8n-006Tco-2O for kvm-riscv@lists.infradead.org; Wed, 19 Apr 2023 22:19:07 +0000 Received: by mail-pl1-x630.google.com with SMTP id d9443c01a7336-1a92369761cso4221475ad.3 for ; Wed, 19 Apr 2023 15:19:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20221208.gappssmtp.com; s=20221208; t=1681942745; x=1684534745; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZFcComAa5TlSFZBz7x8OlktBSnuCSnQ4ODeflbYaEGg=; b=0nUEiKekWfmNI3AIrUphLeoG9IshBvYin/IZirKoihkL7Vz1hKxvtZ9S/+qBRCxzGl Z5d0ha3qHQFVtOuYq101NOgtW6elAbMIjX+yLfEnzAp2xs2I4iBDYY5ZkQhxDmkY/7bK cWsa9maVnCVxVpLIasWERRfsvZygnli2tjaAPy0CiHcpn0ucula1hxhQbTtXkquOE/G9 qpketxd8B8maczWCb6ig5SjaDPKhR+D5kNN9pT45ngLGWR38A7D8mV+x7ZjU7uDYBccF cGzTF21j2WAnsV3IO9kxRpdiOoTA+Oi128benfRj77bo/TAAOLLGK42wViYQ/tssBFno STjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681942745; x=1684534745; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZFcComAa5TlSFZBz7x8OlktBSnuCSnQ4ODeflbYaEGg=; b=jQrVLp/Kr7j5oEIzPV4e4O/si3HgbA1roOYyVpLcWzRBed2LoevEis2PWgVUWjxl9N pzMpSW7FqxUOxlFw0Nbq/zlfTqOivekFKAkEyOQ/q/T8SEYKF0LZYDllbO8P3RCQeUDI B026thTkg9f7UvXZrkjg/4ertX5f/88QuQ3uaN5MVTOVLvc9BlF2yI2oLBEFv/LGsqam FroEhqx4Jht9P56vibUOJwGxtggi3NtLQiPMcvFrcXEN+/drH1Pb6Y9qDy+gtjzj+q6Z iJYgy4S47jclO6Twa4DkR6QGZ1gBVJNMMjaEIgkK16OLfMrXfO0eljjBSYwj28tuaT+7 LCZQ== X-Gm-Message-State: AAQBX9egqeJkpeBh9OQ2Mr1ll6cbs/QyEN9owy0yYLpII3vRJ+I2M7cl rUJL41MlHfHt8jJASZR64CUhjw== X-Google-Smtp-Source: AKy350ZdssyQae53kgooCFIYW6ziRRtfOc1jDxPEdCsn1TsvpfXtraJ7uSS11rn4Y1HrD+jtGXUOIw== X-Received: by 2002:a17:902:c792:b0:19e:7a2c:78a7 with SMTP id w18-20020a170902c79200b0019e7a2c78a7mr5521629pla.57.1681942745174; Wed, 19 Apr 2023 15:19:05 -0700 (PDT) Received: from atishp.ba.rivosinc.com ([66.220.2.162]) by smtp.gmail.com with ESMTPSA id jn11-20020a170903050b00b00196807b5189sm11619190plb.292.2023.04.19.15.19.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 19 Apr 2023 15:19:04 -0700 (PDT) From: Atish Patra To: linux-kernel@vger.kernel.org Cc: Rajnesh Kanwal , Atish Patra , Alexandre Ghiti , Andrew Jones , Andrew Morton , Anup Patel , Atish Patra , =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , Suzuki K Poulose , Will Deacon , Marc Zyngier , Sean Christopherson , linux-coco@lists.linux.dev, Dylan Reid , abrestic@rivosinc.com, Samuel Ortiz , Christoph Hellwig , Conor Dooley , Greg Kroah-Hartman , Guo Ren , Heiko Stuebner , Jiri Slaby , kvm-riscv@lists.infradead.org, kvm@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, Mayuresh Chitale , Palmer Dabbelt , Paolo Bonzini , Paul Walmsley , Uladzislau Rezki Subject: [RFC 43/48] RISC-V: Add base memory encryption functions. Date: Wed, 19 Apr 2023 15:17:11 -0700 Message-Id: <20230419221716.3603068-44-atishp@rivosinc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230419221716.3603068-1-atishp@rivosinc.com> References: <20230419221716.3603068-1-atishp@rivosinc.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230419_151905_780864_49A36A7F X-CRM114-Status: GOOD ( 25.15 ) X-Spam-Score: 0.0 (/) X-Spam-Report: Spam detection software, running on the system "bombadil.infradead.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: From: Rajnesh Kanwal Devices like virtio use shared memory buffers to transfer data. These buffers are part of the guest memory region. For CoVE guest this is not possible as host can not access guest memory. Content analysis details: (0.0 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:630 listed in] [list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid X-BeenThere: kvm-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "kvm-riscv" Errors-To: kvm-riscv-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org From: Rajnesh Kanwal Devices like virtio use shared memory buffers to transfer data. These buffers are part of the guest memory region. For CoVE guest this is not possible as host can not access guest memory. This is solved by VIRTIO_F_ACCESS_PLATFORM feature and SWIOTLB bounce buffers. Guest only allow devices with VIRTIO_F_ACCESS_PLATFORM feature which leads to guest using DMA API and from there moving to SWIOTLB bounce buffer due to SWIOTLB_FORCE flag set for TEE VM. set_memory_encrypted and set_memory_decrypted sit in this allocation path. Based on if a buffer is being decrypted we mark it shared and if it's being encrypted we mark it unshared using hypercalls. Signed-off-by: Rajnesh Kanwal Signed-off-by: Atish Patra --- arch/riscv/Kconfig | 7 ++++ arch/riscv/include/asm/mem_encrypt.h | 26 +++++++++++++ arch/riscv/mm/Makefile | 2 + arch/riscv/mm/init.c | 17 ++++++++- arch/riscv/mm/mem_encrypt.c | 57 ++++++++++++++++++++++++++++ 5 files changed, 108 insertions(+), 1 deletion(-) create mode 100644 arch/riscv/include/asm/mem_encrypt.h create mode 100644 arch/riscv/mm/mem_encrypt.c diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 49c3006..414cee1 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -163,6 +163,11 @@ config ARCH_MMAP_RND_BITS_MAX config ARCH_MMAP_RND_COMPAT_BITS_MAX default 17 +config RISCV_MEM_ENCRYPT + select ARCH_HAS_MEM_ENCRYPT + select ARCH_HAS_FORCE_DMA_UNENCRYPTED + def_bool n + # set if we run in machine mode, cleared if we run in supervisor mode config RISCV_M_MODE bool @@ -515,6 +520,8 @@ config RISCV_COVE_HOST config RISCV_COVE_GUEST bool "Guest Support for Confidential VM Extension(CoVE)" default n + select SWIOTLB + select RISCV_MEM_ENCRYPT help Enables support for running TVMs on platforms supporting CoVE. diff --git a/arch/riscv/include/asm/mem_encrypt.h b/arch/riscv/include/asm/mem_encrypt.h new file mode 100644 index 0000000..0dc3fe8 --- /dev/null +++ b/arch/riscv/include/asm/mem_encrypt.h @@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * RISCV Memory Encryption Support. + * + * Copyright (c) 2023 Rivos Inc. + * + * Authors: + * Rajnesh Kanwal + */ + +#ifndef __RISCV_MEM_ENCRYPT_H__ +#define __RISCV_MEM_ENCRYPT_H__ + +#include + +struct device; + +bool force_dma_unencrypted(struct device *dev); + +/* Architecture __weak replacement functions */ +void __init mem_encrypt_init(void); + +int set_memory_encrypted(unsigned long addr, int numpages); +int set_memory_decrypted(unsigned long addr, int numpages); + +#endif /* __RISCV_MEM_ENCRYPT_H__ */ diff --git a/arch/riscv/mm/Makefile b/arch/riscv/mm/Makefile index 2ac177c..1fd9b60 100644 --- a/arch/riscv/mm/Makefile +++ b/arch/riscv/mm/Makefile @@ -33,3 +33,5 @@ endif obj-$(CONFIG_DEBUG_VIRTUAL) += physaddr.o obj-$(CONFIG_RISCV_DMA_NONCOHERENT) += dma-noncoherent.o + +obj-$(CONFIG_RISCV_MEM_ENCRYPT) += mem_encrypt.o diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index 478d676..b5edd8e 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -21,6 +21,7 @@ #include #include +#include #include #include #include @@ -156,11 +157,25 @@ static void print_vm_layout(void) { } void __init mem_init(void) { + unsigned int flags = SWIOTLB_VERBOSE; + bool swiotlb_en; + + if (is_cove_guest()) { + /* Since the guest memory is inaccessible to the host, devices + * always need to use the SWIOTLB buffer for DMA even if + * dma_capable() says otherwise. + */ + flags |= SWIOTLB_FORCE; + swiotlb_en = true; + } else { + swiotlb_en = !!(max_pfn > PFN_DOWN(dma32_phys_limit)); + } + #ifdef CONFIG_FLATMEM BUG_ON(!mem_map); #endif /* CONFIG_FLATMEM */ - swiotlb_init(max_pfn > PFN_DOWN(dma32_phys_limit), SWIOTLB_VERBOSE); + swiotlb_init(swiotlb_en, flags); memblock_free_all(); print_vm_layout(); diff --git a/arch/riscv/mm/mem_encrypt.c b/arch/riscv/mm/mem_encrypt.c new file mode 100644 index 0000000..8207a5c --- /dev/null +++ b/arch/riscv/mm/mem_encrypt.c @@ -0,0 +1,57 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2023 Rivos Inc. + * + * Authors: + * Rajnesh Kanwal + */ + +#include +#include +#include +#include +#include + +/* Override for DMA direct allocation check - ARCH_HAS_FORCE_DMA_UNENCRYPTED */ +bool force_dma_unencrypted(struct device *dev) +{ + /* + * For authorized devices in trusted guest, all DMA must be to/from + * unencrypted addresses. + */ + return cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT); +} + +int set_memory_encrypted(unsigned long addr, int numpages) +{ + if (!cc_platform_has(CC_ATTR_MEM_ENCRYPT)) + return 0; + + if (!PAGE_ALIGNED(addr)) + return -EINVAL; + + return sbi_covg_unshare_memory(__pa(addr), numpages * PAGE_SIZE); +} +EXPORT_SYMBOL_GPL(set_memory_encrypted); + +int set_memory_decrypted(unsigned long addr, int numpages) +{ + if (!cc_platform_has(CC_ATTR_MEM_ENCRYPT)) + return 0; + + if (!PAGE_ALIGNED(addr)) + return -EINVAL; + + return sbi_covg_share_memory(__pa(addr), numpages * PAGE_SIZE); +} +EXPORT_SYMBOL_GPL(set_memory_decrypted); + +/* Architecture __weak replacement functions */ +void __init mem_encrypt_init(void) +{ + if (!cc_platform_has(CC_ATTR_MEM_ENCRYPT)) + return; + + /* Call into SWIOTLB to update the SWIOTLB DMA buffers */ + swiotlb_update_mem_attributes(); +}