From patchwork Mon Aug 21 06:47:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chengen Du X-Patchwork-Id: 1823550 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=RNQL5m58; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=patchwork.ozlabs.org) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RTjj06MTfz1ybW for ; Mon, 21 Aug 2023 16:47:47 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1qXyhO-0004Co-0T; Mon, 21 Aug 2023 06:47:38 +0000 Received: from smtp-relay-internal-0.internal ([10.131.114.225] helo=smtp-relay-internal-0.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1qXyhL-0004CM-LZ for kernel-team@lists.ubuntu.com; Mon, 21 Aug 2023 06:47:35 +0000 Received: from mail-pj1-f70.google.com (mail-pj1-f70.google.com [209.85.216.70]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 6509D3FAA1 for ; Mon, 21 Aug 2023 06:47:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1692600455; bh=0PscXplakcoQa24slPc6I7rzTnE82nXtivVj9Hgvp4A=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=RNQL5m587n1Y9D0x85y2eXvJHR1dABeggF2f/+N30O9NqBeJq3n149vWiRmxzMYan R+MXckPyLBNQyCRuupuZf60i3fAET2t507dGSGv+ovYPhnzW8FnEs2VI9iefhMoaOt 1qKj435HQko3rw5YOGOnSADemIus2i2bDwGqsp6Mbm8c9efbYWIybNjDcRHDNMpous sLz+jzZi0VyhAGOiosV7+4WttDkAH6+xeVVqnyHvpf2dAMgejPoJl/JPhA7jRytFnQ l+PImXup0uvto0HZFMqh0JMGS13HRbv1MMpfv50Mrgjo13NxPdnx0OuEZ5G4+WAE1M MHo8NNIXBkN8w== Received: by mail-pj1-f70.google.com with SMTP id 98e67ed59e1d1-26d52dc97e3so2507301a91.2 for ; Sun, 20 Aug 2023 23:47:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692600453; x=1693205253; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0PscXplakcoQa24slPc6I7rzTnE82nXtivVj9Hgvp4A=; b=MsEKVQKQxKaBwjCUCKykr7DF3fpT0n2jDSxnzh6eSJFIbGBqC1U75mTfqpbhfFKJLn uqAduDgu5ydKJnbLrqLnoYLQb5sMrIHuJcPJHbslLr7FrkUc8gnW98uQl+0ydeeVvq6t 4DwiqFiHu9OlZhPWx4L3vRvvUEK/LMjxv8QJGbCcs5Dd0ckCaT6ZydBZOUaB3fJgFHi3 iUneH8bFgje0mNrHLn+dgTGYsQdaB0yriN2ZK8ZkUoqTIGxG9ZxntSxkNVk/e3usKIk7 bJRYRY12p+x9EWTg+PDa6byuCje/CyQuj5dsA8/lc169inBewvE82bXhFWT5ENDk3mbo fd4w== X-Gm-Message-State: AOJu0Yxn7uTbkJJ6vR49GpTbWO2plpT+GhDZoCEkRS0Lq4BwAQnu1/O7 YI1zGpKpZ7awiyAi5Ado/5FIFDeN3/ICoO9pFmxai7ar4bwkN8nFWzzEg7RwEfHIKlqZxBcBxOT xZi8KiDhF3LH621jpmvztGqBEsLn/ydk1GOz0DnYcyFSbhifRWQ== X-Received: by 2002:a17:90b:150:b0:26d:287f:a545 with SMTP id em16-20020a17090b015000b0026d287fa545mr2900798pjb.29.1692600453665; Sun, 20 Aug 2023 23:47:33 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGlI+mjDPhKbiNgyZ1Qh2qkqJo6b7FvTdkVA3C97MqKTvyzFBG3wO/Br2ySIr/FIw4mpmWvfg== X-Received: by 2002:a17:90b:150:b0:26d:287f:a545 with SMTP id em16-20020a17090b015000b0026d287fa545mr2900786pjb.29.1692600453205; Sun, 20 Aug 2023 23:47:33 -0700 (PDT) Received: from chengendu.. (111-248-116-169.dynamic-ip.hinet.net. [111.248.116.169]) by smtp.gmail.com with ESMTPSA id l11-20020a17090a598b00b00267b38f5e13sm5275515pji.2.2023.08.20.23.47.32 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Aug 2023 23:47:32 -0700 (PDT) From: Chengen Du To: kernel-team@lists.ubuntu.com Subject: [SRU][J][PATCH 1/2] x86/kvm/fpu: Limit guest user_xfeatures to supported bits of XCR0 Date: Mon, 21 Aug 2023 14:47:27 +0800 Message-Id: <20230821064728.38227-2-chengen.du@canonical.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230821064728.38227-1-chengen.du@canonical.com> References: <20230821064728.38227-1-chengen.du@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Leonardo Bras During host/guest switch (like in kvm_arch_vcpu_ioctl_run()), the kernel swaps the fpu between host/guest contexts, by using fpu_swap_kvm_fpstate(). When xsave feature is available, the fpu swap is done by: - xsave(s) instruction, with guest's fpstate->xfeatures as mask, is used to store the current state of the fpu registers to a buffer. - xrstor(s) instruction, with (fpu_kernel_cfg.max_features & XFEATURE_MASK_FPSTATE) as mask, is used to put the buffer into fpu regs. For xsave(s) the mask is used to limit what parts of the fpu regs will be copied to the buffer. Likewise on xrstor(s), the mask is used to limit what parts of the fpu regs will be changed. The mask for xsave(s), the guest's fpstate->xfeatures, is defined on kvm_arch_vcpu_create(), which (in summary) sets it to all features supported by the cpu which are enabled on kernel config. This means that xsave(s) will save to guest buffer all the fpu regs contents the cpu has enabled when the guest is paused, even if they are not used. This would not be an issue, if xrstor(s) would also do that. xrstor(s)'s mask for host/guest swap is basically every valid feature contained in kernel config, except XFEATURE_MASK_PKRU. Accordingto kernel src, it is instead switched in switch_to() and flush_thread(). Then, the following happens with a host supporting PKRU starts a guest that does not support it: 1 - Host has XFEATURE_MASK_PKRU set. 1st switch to guest, 2 - xsave(s) fpu regs to host fpustate (buffer has XFEATURE_MASK_PKRU) 3 - xrstor(s) guest fpustate to fpu regs (fpu regs have XFEATURE_MASK_PKRU) 4 - guest runs, then switch back to host, 5 - xsave(s) fpu regs to guest fpstate (buffer now have XFEATURE_MASK_PKRU) 6 - xrstor(s) host fpstate to fpu regs. 7 - kvm_vcpu_ioctl_x86_get_xsave() copy guest fpstate to userspace (with XFEATURE_MASK_PKRU, which should not be supported by guest vcpu) On 5, even though the guest does not support PKRU, it does have the flag set on guest fpstate, which is transferred to userspace via vcpu ioctl KVM_GET_XSAVE. This becomes a problem when the user decides on migrating the above guest to another machine that does not support PKRU: the new host restores guest's fpu regs to as they were before (xrstor(s)), but since the new host don't support PKRU, a general-protection exception ocurs in xrstor(s) and that crashes the guest. This can be solved by making the guest's fpstate->user_xfeatures hold a copy of guest_supported_xcr0. This way, on 7 the only flags copied to userspace will be the ones compatible to guest requirements, and thus there will be no issue during migration. As a bonus, it will also fail if userspace tries to set fpu features (with the KVM_SET_XSAVE ioctl) that are not compatible to the guest configuration. Such features will never be returned by KVM_GET_XSAVE or KVM_GET_XSAVE2. Also, since kvm_vcpu_after_set_cpuid() now sets fpstate->user_xfeatures, there is not need to set it in kvm_check_cpuid(). So, change fpstate_realloc() so it does not touch fpstate->user_xfeatures if a non-NULL guest_fpu is passed, which is the case when kvm_check_cpuid() calls it. Signed-off-by: Leonardo Bras Message-Id: <20220217053028.96432-2-leobras@redhat.com> Signed-off-by: Paolo Bonzini (backported from commit ad856280ddea3401e1f5060ef20e6de9f6122c76) [chengen - remove guest_fpu judgement as it's consistently NULL in the context] Signed-off-by: Chengen Du --- arch/x86/kvm/cpuid.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index 528437e3e2f3..8e24e2b22948 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -187,6 +187,8 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) best->ecx |= XFEATURE_MASK_FPSSE; } + vcpu->arch.guest_fpu.fpstate->user_xfeatures = vcpu->arch.guest_supported_xcr0; + kvm_update_pv_runtime(vcpu); vcpu->arch.maxphyaddr = cpuid_query_maxphyaddr(vcpu); From patchwork Mon Aug 21 06:47:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chengen Du X-Patchwork-Id: 1823551 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=sMsUhOEp; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=patchwork.ozlabs.org) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RTjj06zz5z1ygk for ; Mon, 21 Aug 2023 16:47:47 +1000 (AEST) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1qXyhP-0004D5-7i; Mon, 21 Aug 2023 06:47:39 +0000 Received: from smtp-relay-internal-0.internal ([10.131.114.225] helo=smtp-relay-internal-0.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1qXyhM-0004CT-OA for kernel-team@lists.ubuntu.com; Mon, 21 Aug 2023 06:47:36 +0000 Received: from mail-pj1-f72.google.com (mail-pj1-f72.google.com [209.85.216.72]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-0.canonical.com (Postfix) with ESMTPS id 961E93FAA1 for ; Mon, 21 Aug 2023 06:47:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1692600456; bh=i2V6Pz1Rtd523sDqwuqkVOiZ+5cEVFmxSZRvsEN/bjo=; h=From:To:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=sMsUhOEp0+VNlz/XUrw5JI/b5AJsw/SsXFsQTpwFAQp3FGPmfinxmUsxUqUl/UrG7 JKseyvYZT7Kwg4O7UT4ZXvWxSxBNAQP3cyQQ5WpUbdU6h9C57mkpTwh0W+G7uaKN2K VBRbZdetORpzucNtU8+pRGIO0hlUFtzU8IOVUFUU2wrKRTDC4hsUKYEUKtoB+Bg3qj DqDWZX34WHxFMzOMUUkBBnjWK8rf8O8J2PyoFVrn30Pu9vgk6bAjxOhA/hIFqwXhB1 1JZAT061bqyp8kKQxarB4tB4HxmQHYxWuA8NXoumd3u+rSRzCZERNjbz3IytNxOH+j +/SbiOY0W/YmQ== Received: by mail-pj1-f72.google.com with SMTP id 98e67ed59e1d1-267f00f6876so3141590a91.3 for ; Sun, 20 Aug 2023 23:47:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692600455; x=1693205255; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=i2V6Pz1Rtd523sDqwuqkVOiZ+5cEVFmxSZRvsEN/bjo=; b=UK6QWDFnHoxm6Y/T6UQe8/zkag/mkuLIlsogD8Gjp7ZQOChlYFDcgCw6g3oggfoPUc IgYOIuQ5A1pk6vY4lR6x0Z3iOeBYOf8k7m3TbCOnJPjC/lG80jPob0f2SadeC3jLHrlE iZwyKucoEc2krWPFeWu2M06GJYf0rQ/yoUay47Xh4p4WL0dlPEm6fxkehMyGqfDnAIi5 vOHrYbexQhHM7DkyavqQKc9CeyNSd3KNEDJxpp9oD2NNY3GV9jDGmo/nNO+P8qRxJO6r tn2ajDnmV+5uvpPZflwRh68zdf51TH3mRCHqfFUytok8UKISNsjazoQM+0sOOQyisb65 BGVQ== X-Gm-Message-State: AOJu0YyJEH0qmB39oWBhpim+R9WcLy37bH8nbePhe8/wzNACM59FRC+D sYMyECaGPwUwKuo3SswNj9N4zLMkqJoJUo+y/TXOhUY0U8/Ax2JHP+3luqva4q5kBz66Zq6Fi9K 8ZvLlF1hmH7Xrz/FEcIc2j2tew/Jl+lwDVQFFpkF81IbN24UGjA== X-Received: by 2002:a17:90a:ce17:b0:262:f579:41db with SMTP id f23-20020a17090ace1700b00262f57941dbmr2853661pju.6.1692600454950; Sun, 20 Aug 2023 23:47:34 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGk8uT/PWWjfRTxxsSh5DKa/dEtI/kPDn20fN+ypD53dUpQhmVi0GtRrGcBCSLmwlP0/y3HQA== X-Received: by 2002:a17:90a:ce17:b0:262:f579:41db with SMTP id f23-20020a17090ace1700b00262f57941dbmr2853655pju.6.1692600454627; Sun, 20 Aug 2023 23:47:34 -0700 (PDT) Received: from chengendu.. (111-248-116-169.dynamic-ip.hinet.net. [111.248.116.169]) by smtp.gmail.com with ESMTPSA id l11-20020a17090a598b00b00267b38f5e13sm5275515pji.2.2023.08.20.23.47.33 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 20 Aug 2023 23:47:34 -0700 (PDT) From: Chengen Du To: kernel-team@lists.ubuntu.com Subject: [SRU][J][PATCH 2/2] KVM: x86: Always enable legacy FP/SSE in allowed user XFEATURES Date: Mon, 21 Aug 2023 14:47:28 +0800 Message-Id: <20230821064728.38227-3-chengen.du@canonical.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230821064728.38227-1-chengen.du@canonical.com> References: <20230821064728.38227-1-chengen.du@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: "Dr. David Alan Gilbert" Allow FP and SSE state to be saved and restored via KVM_{G,SET}_XSAVE on XSAVE-capable hosts even if their bits are not exposed to the guest via XCR0. Failing to allow FP+SSE first showed up as a QEMU live migration failure, where migrating a VM from a pre-XSAVE host, e.g. Nehalem, to an XSAVE host failed due to KVM rejecting KVM_SET_XSAVE. However, the bug also causes problems even when migrating between XSAVE-capable hosts as KVM_GET_SAVE won't set any bits in user_xfeatures if XSAVE isn't exposed to the guest, i.e. KVM will fail to actually migrate FP+SSE. Because KVM_{G,S}ET_XSAVE are designed to allowing migrating between hosts with and without XSAVE, KVM_GET_XSAVE on a non-XSAVE (by way of fpu_copy_guest_fpstate_to_uabi()) always sets the FP+SSE bits in the header so that KVM_SET_XSAVE will work even if the new host supports XSAVE. Fixes: ad856280ddea ("x86/kvm/fpu: Limit guest user_xfeatures to supported bits of XCR0") bz: https://bugzilla.redhat.com/show_bug.cgi?id=2079311 Cc: stable@vger.kernel.org Cc: Leonardo Bras Signed-off-by: Dr. David Alan Gilbert [sean: add comment, massage changelog] Signed-off-by: Sean Christopherson Message-Id: <20220824033057.3576315-3-seanjc@google.com> Signed-off-by: Paolo Bonzini (cherry picked from commit a1020a25e69755a8a1a37735d674b91d6f02939f) Signed-off-by: Chengen Du --- arch/x86/kvm/cpuid.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index 8e24e2b22948..d9cd49305c84 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -187,7 +187,13 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) best->ecx |= XFEATURE_MASK_FPSSE; } - vcpu->arch.guest_fpu.fpstate->user_xfeatures = vcpu->arch.guest_supported_xcr0; + /* + * FP+SSE can always be saved/restored via KVM_{G,S}ET_XSAVE, even if + * XSAVE/XCRO are not exposed to the guest, and even if XSAVE isn't + * supported by the host. + */ + vcpu->arch.guest_fpu.fpstate->user_xfeatures = vcpu->arch.guest_supported_xcr0 | + XFEATURE_MASK_FPSSE; kvm_update_pv_runtime(vcpu);