From patchwork Wed Jul 10 20:23:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brijesh Singh X-Patchwork-Id: 1130567 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=amd.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=amdcloud.onmicrosoft.com header.i=@amdcloud.onmicrosoft.com header.b="oYaJT7i6"; dkim-atps=neutral Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 45kWBQ0Q8Sz9s00 for ; Thu, 11 Jul 2019 06:34:22 +1000 (AEST) Received: from localhost ([::1]:36918 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hlJIC-0006FO-6I for incoming@patchwork.ozlabs.org; Wed, 10 Jul 2019 16:34:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:57307) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hlJ7Y-0001fq-6E for qemu-devel@nongnu.org; Wed, 10 Jul 2019 16:23:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hlJ7V-0002dv-Rq for qemu-devel@nongnu.org; Wed, 10 Jul 2019 16:23:19 -0400 Received: from mail-eopbgr790080.outbound.protection.outlook.com ([40.107.79.80]:1472 helo=NAM03-CO1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hlJ7U-0002MZ-18 for qemu-devel@nongnu.org; Wed, 10 Jul 2019 16:23:16 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector1-amdcloud-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hVZwyG19gNDYgzLx26Rmjpd9gBSMifIOcZmiFf05UYg=; b=oYaJT7i6XwrR6i8Lu+uZqVtZeVr40LDXhoszDRntAHkWxWcRt1Qn60TpHnXQQUsVX9dAAlo9qV7u02HJge42cYXyBs9DWlmMPGgY9kC1Cdb2hCuyypK/oeFCxWiJ4Jn9TDJHC8mtzCQdtC2Jxb6NyV49xRH5rDeA5rqyiUvK0pg= Received: from DM6PR12MB2682.namprd12.prod.outlook.com (20.176.116.31) by DM6PR12MB3820.namprd12.prod.outlook.com (10.255.173.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2052.18; Wed, 10 Jul 2019 20:23:10 +0000 Received: from DM6PR12MB2682.namprd12.prod.outlook.com ([fe80::bc1a:a30d:9da2:1cdd]) by DM6PR12MB2682.namprd12.prod.outlook.com ([fe80::bc1a:a30d:9da2:1cdd%6]) with mapi id 15.20.2073.008; Wed, 10 Jul 2019 20:23:10 +0000 From: "Singh, Brijesh" To: "qemu-devel@nongnu.org" Thread-Topic: [PATCH v2 12/13] migration: add support to migrate page encryption bitmap Thread-Index: AQHVN11JNOx08kq+nEiBgv3DRmN3NA== Date: Wed, 10 Jul 2019 20:23:08 +0000 Message-ID: <20190710202219.25939-13-brijesh.singh@amd.com> References: <20190710202219.25939-1-brijesh.singh@amd.com> In-Reply-To: <20190710202219.25939-1-brijesh.singh@amd.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: DM3PR11CA0020.namprd11.prod.outlook.com (2603:10b6:0:54::30) To DM6PR12MB2682.namprd12.prod.outlook.com (2603:10b6:5:4a::31) authentication-results: spf=none (sender IP is ) smtp.mailfrom=brijesh.singh@amd.com; x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.17.1 x-originating-ip: [165.204.77.1] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: f9a90104-272e-440b-399b-08d705746c51 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020); SRVR:DM6PR12MB3820; x-ms-traffictypediagnostic: DM6PR12MB3820: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:556; x-forefront-prvs: 0094E3478A x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(366004)(346002)(39860400002)(136003)(376002)(396003)(189003)(199004)(256004)(8936002)(2616005)(1076003)(316002)(14444005)(476003)(81166006)(478600001)(54906003)(81156014)(2351001)(8676002)(486006)(25786009)(6436002)(6486002)(4326008)(66476007)(66946007)(446003)(53936002)(50226002)(71190400001)(66446008)(68736007)(6506007)(14454004)(386003)(5660300002)(36756003)(66066001)(186003)(6512007)(99286004)(305945005)(102836004)(26005)(2501003)(11346002)(64756008)(52116002)(5640700003)(66556008)(76176011)(86362001)(6116002)(6916009)(3846002)(7736002)(71200400001)(2906002); DIR:OUT; SFP:1101; SCL:1; SRVR:DM6PR12MB3820; H:DM6PR12MB2682.namprd12.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: amd.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: 83dsrjFKSd/B9z+CJqN+R5429XTTna6o/hbvcXj++gGhaWmSJJB4eLXM10egq2JiK1Nf8huCVjRYTyOe9ZKLq/J+UAol5mYbld13pMtrAUKZLlPMFTi2GhCAdnfyEs3dnZfMulw56tqoDFMgy8LsSpQmApGshfqpvHzCCO+Q4FFnW5eHI8rsVttQVzHWXikBjxhBpNbrN5vORHTHDI3Gh0XLEg4AbHRzJmSaJYcUJnkjmBwROMEsV9lcOk36TBpkdSgYn+I9uKlYUHKTFpinf4wyu5Ds+HFvbDsfhYOna8m8wGY4C5/JMkMuKh8CysfLWk6zEI3raUGPf1+VZgi+A3WEfmFY+y/wtLoqQmvIxpNicbo/RLw8L7hrVu32ALOjiP+t5e7GfxZasu9lIW2OIsN1peK2WJJSKzS2K/zh7XE= MIME-Version: 1.0 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: f9a90104-272e-440b-399b-08d705746c51 X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Jul 2019 20:23:08.9291 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: sbrijesh@amd.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB3820 X-detected-operating-system: by eggs.gnu.org: Windows 7 or 8 [fuzzy] X-Received-From: 40.107.79.80 Subject: [Qemu-devel] [PATCH v2 12/13] migration: add support to migrate page encryption bitmap X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "pbonzini@redhat.com" , "Lendacky, Thomas" , "Singh, Brijesh" , "dgilbert@redhat.com" , "ehabkost@redhat.com" Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" When memory encryption is enabled, the hypervisor maintains a page encryption bitmap which is referred by hypervisor during migratoin to check if page is private or shared. The bitmap is built during the VM bootup and must be migrated to the target host so that hypervisor on target host can use it for future migration. The KVM_{SET,GET}_PAGE_ENC_BITMAP can be used to get and set the bitmap for a given gfn range. Signed-off-by: Brijesh Singh --- accel/kvm/kvm-all.c | 4 +++ migration/ram.c | 11 +++++++ target/i386/sev.c | 67 ++++++++++++++++++++++++++++++++++++++++ target/i386/trace-events | 2 ++ 4 files changed, 84 insertions(+) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index 442b1af36e..9e23088a94 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -1831,6 +1831,10 @@ static int kvm_init(MachineState *ms) kvm_state->memcrypt_encrypt_data = sev_encrypt_data; kvm_state->memcrypt_save_outgoing_page = sev_save_outgoing_page; kvm_state->memcrypt_load_incoming_page = sev_load_incoming_page; + kvm_state->memcrypt_load_incoming_page_enc_bitmap = + sev_load_incoming_page_enc_bitmap; + kvm_state->memcrypt_save_outgoing_page_enc_bitmap = + sev_save_outgoing_page_enc_bitmap; } ret = kvm_arch_init(ms, s); diff --git a/migration/ram.c b/migration/ram.c index d179867e1b..3a4bdf3c03 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -78,6 +78,7 @@ /* 0x80 is reserved in migration.h start with 0x100 next */ #define RAM_SAVE_FLAG_COMPRESS_PAGE 0x100 #define RAM_SAVE_FLAG_ENCRYPTED_PAGE 0x200 +#define RAM_SAVE_FLAG_PAGE_ENCRYPTED_BITMAP 0x400 /* used in target/i386/sev.c */ static inline bool is_zero_range(uint8_t *p, uint64_t size) { @@ -3595,6 +3596,10 @@ static int ram_save_complete(QEMUFile *f, void *opaque) flush_compressed_data(rs); ram_control_after_iterate(f, RAM_CONTROL_FINISH); + if (kvm_memcrypt_enabled()) { + ret = kvm_memcrypt_save_outgoing_page_enc_bitmap(f); + } + rcu_read_unlock(); multifd_send_sync_main(); @@ -4469,6 +4474,12 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id) ret = -EINVAL; } break; + case RAM_SAVE_FLAG_PAGE_ENCRYPTED_BITMAP: + if (kvm_memcrypt_load_incoming_page_enc_bitmap(f)) { + error_report("Failed to load page enc bitmap"); + ret = -EINVAL; + } + break; case RAM_SAVE_FLAG_EOS: /* normal exit */ multifd_recv_sync_main(); diff --git a/target/i386/sev.c b/target/i386/sev.c index 09a62d6f88..93c6a90806 100644 --- a/target/i386/sev.c +++ b/target/i386/sev.c @@ -63,6 +63,7 @@ static const char *const sev_fw_errlist[] = { }; #define SEV_FW_MAX_ERROR ARRAY_SIZE(sev_fw_errlist) +#define RAM_SAVE_FLAG_PAGE_ENCRYPTED_BITMAP 0x400 static int sev_ioctl(int fd, int cmd, void *data, int *error) @@ -1189,6 +1190,72 @@ int sev_load_incoming_page(void *handle, QEMUFile *f, uint8_t *ptr) return sev_receive_update_data(f, ptr); } +#define ALIGN(x, y) (((x)+(y)-1) & ~((y)-1)) + +int sev_load_incoming_page_enc_bitmap(void *handle, QEMUFile *f) +{ + void *bmap; + unsigned long npages; + unsigned long bmap_size, base_gpa; + struct kvm_page_enc_bitmap e = {}; + + base_gpa = qemu_get_be64(f); + npages = qemu_get_be64(f); + bmap_size = qemu_get_be64(f); + + bmap = g_malloc0(bmap_size); + qemu_get_buffer(f, (uint8_t *)bmap, bmap_size); + + trace_kvm_sev_load_page_enc_bitmap(base_gpa, npages << TARGET_PAGE_BITS); + + e.start_gfn = base_gpa >> TARGET_PAGE_BITS; + e.num_pages = npages; + e.enc_bitmap = bmap; + if (kvm_vm_ioctl(kvm_state, KVM_SET_PAGE_ENC_BITMAP, &e) == -1) { + error_report("KVM_SET_PAGE_ENC_BITMAP ioctl failed %d", errno); + g_free(bmap); + return 1; + } + + g_free(bmap); + + return 0; +} + +int sev_save_outgoing_page_enc_bitmap(void *handle, QEMUFile *f, + unsigned long start, uint64_t length) +{ + uint64_t size; + struct kvm_page_enc_bitmap e = {}; + + if (!length) { + return 0; + } + + size = ALIGN((length >> TARGET_PAGE_BITS), /*HOST_LONG_BITS*/ 64) / 8; + e.enc_bitmap = g_malloc0(size); + e.start_gfn = start >> TARGET_PAGE_BITS; + e.num_pages = length >> TARGET_PAGE_BITS; + + trace_kvm_sev_save_page_enc_bitmap(start, length); + + if (kvm_vm_ioctl(kvm_state, KVM_GET_PAGE_ENC_BITMAP, &e) == -1) { + error_report("%s: KVM_GET_PAGE_ENC_BITMAP ioctl failed %d", + __func__, errno); + g_free(e.enc_bitmap); + return 1; + } + + qemu_put_be64(f, RAM_SAVE_FLAG_PAGE_ENCRYPTED_BITMAP); + qemu_put_be64(f, start); + qemu_put_be64(f, e.num_pages); + qemu_put_be64(f, size); + qemu_put_buffer(f, (uint8_t *)e.enc_bitmap, size); + + g_free(e.enc_bitmap); + return 0; +} + static void sev_register_types(void) { diff --git a/target/i386/trace-events b/target/i386/trace-events index 609752cca7..4c2be570f9 100644 --- a/target/i386/trace-events +++ b/target/i386/trace-events @@ -21,3 +21,5 @@ kvm_sev_send_finish(void) "" kvm_sev_receive_start(int policy, void *session, void *pdh) "policy 0x%x session %p pdh %p" kvm_sev_receive_update_data(void *src, void *dst, int len, void *hdr, int hdr_len) "guest %p trans %p len %d hdr %p hdr_len %d" kvm_sev_receive_finish(void) "" +kvm_sev_save_page_enc_bitmap(uint64_t start, uint64_t len) "start 0x%" PRIx64 " len 0x%" PRIx64 +kvm_sev_load_page_enc_bitmap(uint64_t start, uint64_t len) "start 0x%" PRIx64 " len 0x%" PRIx64