From patchwork Wed May 3 10:52:19 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Xiao Guangrong X-Patchwork-Id: 757952 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3wHw7k05k9z9rvt for ; Wed, 3 May 2017 20:56:18 +1000 (AEST) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="l7XLT+uO"; dkim-atps=neutral Received: from localhost ([::1]:35779 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d5rx9-0007Dw-Fr for incoming@patchwork.ozlabs.org; Wed, 03 May 2017 06:56:15 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:32959) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d5ruC-0005Gv-Mt for qemu-devel@nongnu.org; Wed, 03 May 2017 06:53:14 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d5ruB-0006h1-J6 for qemu-devel@nongnu.org; Wed, 03 May 2017 06:53:12 -0400 Received: from mail-pf0-x241.google.com ([2607:f8b0:400e:c00::241]:35939) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1d5ruB-0006gc-BG for qemu-devel@nongnu.org; Wed, 03 May 2017 06:53:11 -0400 Received: by mail-pf0-x241.google.com with SMTP id v14so3779097pfd.3 for ; Wed, 03 May 2017 03:53:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=w7qCcA41saMZCVP0zlh5DQ474ttL6rR3xS+/9x5IAcg=; b=l7XLT+uOsni7lZ2yYYX8OP1KsVOru5ovvXlUApIl5kHdajZx9zKRDgJSzzyRvtnOLG O2V5gkfHFvGUyrvXp0nMltFsEbSZo5Evu9TvAd92Sme90Q2M1U08E+dARJZ617ctesxH +PYME90gjCkqge3PiCAwWYV7ToD/CYgnJk7uvbg6GcICzNqNThCfPHZ2ngrZtA9niZg5 783EeiTXtk0F//YiiCyM0cpmE+75VL0BEJ9xClH0icIIDP4QFIR3XJRA27KGmhRYPNjp RW/s35bkoOnhsPj8GN9vyTd4NCE26FWzwJ92hKWL4UybxFdqt1Fon6/SYioO2l1/SBK1 TGWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=w7qCcA41saMZCVP0zlh5DQ474ttL6rR3xS+/9x5IAcg=; b=pirjlhJouFwm4Ubrcd/Vvxl3cXBxzr0iMlKWySIybB6qhauaosAXISNGR9w5GUND9T 2BHv3yS3/d4nme1lhVmdG21TswZy4J61i8Eb1mx+XvWbgq/qI2CuU2OVOK3Dbf15Y4VD DzGPvH8DxsfuQNAa7kXbg7Hw1g1hpv8YAnPN8KCrEvphuSon1l0E8bp1pitLowtOTU8c 1O2LXUAsHIIW7tol1hrNnDalc0UtCl5Ceot3Uyptjt2eBiH5mEjFRT6hduKDMsHr4HZl NRuL/3LP8WHLn7IV8eDhxy7buKMbRBwvi8CE8x3zEHWVF9V/ewQR4muoGMNmy7OvhmgC VSzg== X-Gm-Message-State: AN3rC/73H+4s7K/nDz5rgGTzBJ0MjjVSJtzHVlxRU31q1fQ5x2WW9CE+ 6ywSAHYjVH1O2A== X-Received: by 10.98.149.196 with SMTP id c65mr4157853pfk.37.1493808790324; Wed, 03 May 2017 03:53:10 -0700 (PDT) Received: from eric.tencent.com ([203.205.141.35]) by smtp.gmail.com with ESMTPSA id d24sm4395561pfb.97.2017.05.03.03.53.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 03 May 2017 03:53:09 -0700 (PDT) From: guangrong.xiao@gmail.com X-Google-Original-From: xiaoguangrong@tencent.com To: pbonzini@redhat.com, mtosatti@redhat.com, avi.kivity@gmail.com, rkrcmar@redhat.com Date: Wed, 3 May 2017 18:52:19 +0800 Message-Id: <20170503105224.19049-3-xiaoguangrong@tencent.com> X-Mailer: git-send-email 2.9.3 In-Reply-To: <20170503105224.19049-1-xiaoguangrong@tencent.com> References: <20170503105224.19049-1-xiaoguangrong@tencent.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::241 Subject: [Qemu-devel] [PATCH 2/7] KVM: MMU: introduce possible_writable_spte_bitmap X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Xiao Guangrong , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" From: Xiao Guangrong It is used to track possible writable sptes on the shadow page on which the bit is set to 1 for the sptes that are already writable or can be locklessly updated to writable on the fast_page_fault path, also a counter for the number of possible writable sptes is introduced to speed up bitmap walking Later patch will benefit good performance by using this bitmap and counter to fast figure out writable sptes and write protect them Signed-off-by: Xiao Guangrong --- arch/x86/include/asm/kvm_host.h | 6 ++++- arch/x86/kvm/mmu.c | 53 ++++++++++++++++++++++++++++++++++++++++- 2 files changed, 57 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 84c8489..4872ae7 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -114,6 +114,7 @@ static inline gfn_t gfn_to_index(gfn_t gfn, gfn_t base_gfn, int level) #define KVM_MIN_ALLOC_MMU_PAGES 64 #define KVM_MMU_HASH_SHIFT 12 #define KVM_NUM_MMU_PAGES (1 << KVM_MMU_HASH_SHIFT) +#define KVM_MMU_SP_ENTRY_NR 512 #define KVM_MIN_FREE_MMU_PAGES 5 #define KVM_REFILL_PAGES 25 #define KVM_MAX_CPUID_ENTRIES 80 @@ -287,12 +288,15 @@ struct kvm_mmu_page { bool unsync; int root_count; /* Currently serving as active root */ unsigned int unsync_children; + unsigned int possiable_writable_sptes; struct kvm_rmap_head parent_ptes; /* rmap pointers to parent sptes */ /* The page is obsolete if mmu_valid_gen != kvm->arch.mmu_valid_gen. */ unsigned long mmu_valid_gen; - DECLARE_BITMAP(unsync_child_bitmap, 512); + DECLARE_BITMAP(unsync_child_bitmap, KVM_MMU_SP_ENTRY_NR); + + DECLARE_BITMAP(possible_writable_spte_bitmap, KVM_MMU_SP_ENTRY_NR); #ifdef CONFIG_X86_32 /* diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c index ba8e7af..8a20e4f 100644 --- a/arch/x86/kvm/mmu.c +++ b/arch/x86/kvm/mmu.c @@ -570,6 +570,49 @@ static bool is_dirty_spte(u64 spte) : spte & PT_WRITABLE_MASK; } +static bool is_possible_writable_spte(u64 spte) +{ + if (!is_shadow_present_pte(spte)) + return false; + + if (is_writable_pte(spte)) + return true; + + if (spte_can_locklessly_be_made_writable(spte)) + return true; + + /* + * although is_access_track_spte() sptes can be updated out of + * mmu-lock, we need not take them into account as access_track + * drops writable bit for them + */ + return false; +} + +static void +mmu_log_possible_writable_spte(u64 *sptep, u64 old_spte, u64 new_spte) +{ + struct kvm_mmu_page *sp = page_header(__pa(sptep)); + bool old_state, new_state; + + old_state = is_possible_writable_spte(old_spte); + new_state = is_possible_writable_spte(new_spte); + + if (old_state == new_state) + return; + + /* a possible writable spte is dropped */ + if (old_state) { + sp->possiable_writable_sptes--; + __clear_bit(sptep - sp->spt, sp->possible_writable_spte_bitmap); + return; + } + + /* a new possible writable spte is set */ + sp->possiable_writable_sptes++; + __set_bit(sptep - sp->spt, sp->possible_writable_spte_bitmap); +} + /* Rules for using mmu_spte_set: * Set the sptep from nonpresent to present. * Note: the sptep being assigned *must* be either not present @@ -580,6 +623,7 @@ static void mmu_spte_set(u64 *sptep, u64 new_spte) { WARN_ON(is_shadow_present_pte(*sptep)); __set_spte(sptep, new_spte); + mmu_log_possible_writable_spte(sptep, 0ull, new_spte); } /* @@ -598,6 +642,7 @@ static void mmu_spte_update_no_track(u64 *sptep, u64 new_spte) } __update_clear_spte_fast(sptep, new_spte); + mmu_log_possible_writable_spte(sptep, old_spte, new_spte); } /* @@ -623,6 +668,7 @@ static u64 mmu_spte_update_track(u64 *sptep, u64 new_spte) WARN_ON(spte_to_pfn(old_spte) != spte_to_pfn(new_spte)); + mmu_log_possible_writable_spte(sptep, old_spte, new_spte); return old_spte; } @@ -688,6 +734,8 @@ static int mmu_spte_clear_track_bits(u64 *sptep) else old_spte = __update_clear_spte_slow(sptep, 0ull); + mmu_log_possible_writable_spte(sptep, old_spte, 0ull); + if (!is_shadow_present_pte(old_spte)) return 0; @@ -716,7 +764,10 @@ static int mmu_spte_clear_track_bits(u64 *sptep) */ static void mmu_spte_clear_no_track(u64 *sptep) { + u64 old_spte = *sptep; + __update_clear_spte_fast(sptep, 0ull); + mmu_log_possible_writable_spte(sptep, old_spte, 0ull); } static u64 mmu_spte_get_lockless(u64 *sptep) @@ -1988,7 +2039,7 @@ static int __mmu_unsync_walk(struct kvm_mmu_page *sp, { int i, ret, nr_unsync_leaf = 0; - for_each_set_bit(i, sp->unsync_child_bitmap, 512) { + for_each_set_bit(i, sp->unsync_child_bitmap, KVM_MMU_SP_ENTRY_NR) { struct kvm_mmu_page *child; u64 ent = sp->spt[i];