From patchwork Fri Aug 7 17:03:08 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alvise Rigo X-Patchwork-Id: 505198 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 357A9140285 for ; Sat, 8 Aug 2015 03:01:30 +1000 (AEST) Received: from localhost ([::1]:49987 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZNl1M-0003ny-6o for incoming@patchwork.ozlabs.org; Fri, 07 Aug 2015 13:01:28 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36210) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZNl0Z-0002ZO-Q9 for qemu-devel@nongnu.org; Fri, 07 Aug 2015 13:00:46 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZNl0V-0004D5-RN for qemu-devel@nongnu.org; Fri, 07 Aug 2015 13:00:38 -0400 Received: from mail-wi0-f171.google.com ([209.85.212.171]:37597) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZNl0V-0004Cu-Dl for qemu-devel@nongnu.org; Fri, 07 Aug 2015 13:00:35 -0400 Received: by wibhh20 with SMTP id hh20so73618931wib.0 for ; Fri, 07 Aug 2015 10:00:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=wPmpgq/VH1LPhKA5XFIsexynqNMDfUpqX2akiXvp0Yo=; b=Cb9uq6H6FfP6rU9ClbgvHwhwiLbiSv+jLacM093oMeY3KwrSXSpyL9qr7sKsI0m1Y/ yJJzKnSKXPnXQQASo3UXnlmrR1BWgTzrH3eEq3VCqXfpAk0zYbT90q6hpzCdqUdByEoP jy3kS/BJ6xJkIKoI4DQ+1iMHkIbI+OavNg96cv7Hz6txr69Zi52HWJAZesTc95y7NRp9 /pH8ly+z97dZjCYieMArh7ea8B7Ui4aJnsCnMDLh0JKDKExPgrjqJ9v6Z28vsWSX0Gs2 56GZgAWus6p6Dzk9z3jDAcckkO9CqFHaaDX1v6zAJRAJWmcn/OGbGBW1hcnOezUKkIH8 ek0g== X-Gm-Message-State: ALoCoQlpfzvmxZ/kRrGi4Vz6d3FwDXkP2szsaRfhpL/7BvsMw32D1otdPnPy8PEzFXCvyaPEiRjM X-Received: by 10.180.23.69 with SMTP id k5mr9350640wif.3.1438966834856; Fri, 07 Aug 2015 10:00:34 -0700 (PDT) Received: from linarch.home (LPuteaux-656-1-278-113.w80-15.abo.wanadoo.fr. [80.15.154.113]) by smtp.googlemail.com with ESMTPSA id c7sm15490379wjb.19.2015.08.07.10.00.34 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 07 Aug 2015 10:00:34 -0700 (PDT) From: Alvise Rigo To: qemu-devel@nongnu.org, mttcg@listserver.greensocs.com Date: Fri, 7 Aug 2015 19:03:08 +0200 Message-Id: <1438966995-5913-3-git-send-email-a.rigo@virtualopensystems.com> X-Mailer: git-send-email 2.5.0 In-Reply-To: <1438966995-5913-1-git-send-email-a.rigo@virtualopensystems.com> References: <1438966995-5913-1-git-send-email-a.rigo@virtualopensystems.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.85.212.171 Cc: alex.bennee@linaro.org, jani.kokkonen@huawei.com, tech@virtualopensystems.com, claudio.fontana@huawei.com, pbonzini@redhat.com Subject: [Qemu-devel] [RFC v4 2/9] softmmu: Add new TLB_EXCL flag X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Add a new TLB flag to force all the accesses made to a page to follow the slow-path. In the case we remove a TLB entry marked as EXCL, we unset the corresponding exclusive bit in the bitmap. Suggested-by: Jani Kokkonen Suggested-by: Claudio Fontana Signed-off-by: Alvise Rigo --- cputlb.c | 39 ++++++++++++++++- include/exec/cpu-all.h | 8 ++++ include/exec/cpu-defs.h | 12 ++++++ softmmu_template.h | 112 ++++++++++++++++++++++++++++++++++++++---------- 4 files changed, 148 insertions(+), 23 deletions(-) diff --git a/cputlb.c b/cputlb.c index a506086..251eec8 100644 --- a/cputlb.c +++ b/cputlb.c @@ -299,6 +299,13 @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr, env->tlb_v_table[mmu_idx][vidx] = *te; env->iotlb_v[mmu_idx][vidx] = env->iotlb[mmu_idx][index]; + if (unlikely(!(te->addr_write & TLB_MMIO) && (te->addr_write & TLB_EXCL))) { + /* We are removing an exclusive entry, set the page to dirty. */ + hwaddr hw_addr = (env->iotlb[mmu_idx][index].addr & TARGET_PAGE_MASK) + + (te->addr_write & TARGET_PAGE_MASK); + cpu_physical_memory_set_excl_dirty(hw_addr, cpu->cpu_index); + } + /* refill the tlb */ env->iotlb[mmu_idx][index].addr = iotlb - vaddr; env->iotlb[mmu_idx][index].attrs = attrs; @@ -324,7 +331,15 @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr, + xlat)) { te->addr_write = address | TLB_NOTDIRTY; } else { - te->addr_write = address; + if (!(address & TLB_MMIO) && + cpu_physical_memory_excl_atleast_one_clean(section->mr->ram_addr + + xlat)) { + /* There is at least one vCPU that has flagged the address as + * exclusive. */ + te->addr_write = address | TLB_EXCL; + } else { + te->addr_write = address; + } } } else { te->addr_write = -1; @@ -376,6 +391,28 @@ tb_page_addr_t get_page_addr_code(CPUArchState *env1, target_ulong addr) return qemu_ram_addr_from_host_nofail(p); } +/* Atomic insn translation TLB support. */ +#define EXCLUSIVE_RESET_ADDR ULLONG_MAX +/* For every vCPU compare the exclusive address and reset it in case of a + * match. Since only one vCPU is running at once, no lock has to be held to + * guard this operation. */ +static inline void lookup_and_reset_cpus_ll_addr(hwaddr addr, hwaddr size) +{ + CPUState *cpu; + CPUArchState *acpu; + + CPU_FOREACH(cpu) { + acpu = (CPUArchState *)cpu->env_ptr; + + if (acpu->excl_protected_range.begin != EXCLUSIVE_RESET_ADDR && + ranges_overlap(acpu->excl_protected_range.begin, + acpu->excl_protected_range.end - acpu->excl_protected_range.begin, + addr, size)) { + acpu->excl_protected_range.begin = EXCLUSIVE_RESET_ADDR; + } + } +} + #define MMUSUFFIX _mmu #define SHIFT 0 diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h index ea6a9a6..ad6afcb 100644 --- a/include/exec/cpu-all.h +++ b/include/exec/cpu-all.h @@ -320,6 +320,14 @@ extern RAMList ram_list; #define TLB_NOTDIRTY (1 << 4) /* Set if TLB entry is an IO callback. */ #define TLB_MMIO (1 << 5) +/* Set if TLB entry references a page that requires exclusive access. */ +#define TLB_EXCL (1 << 6) + +/* Do not allow a TARGET_PAGE_MASK which covers one or more bits defined + * above. */ +#if TLB_EXCL >= TARGET_PAGE_SIZE +#error TARGET_PAGE_MASK covering the low bits of the TLB virtual address +#endif void dump_exec_info(FILE *f, fprintf_function cpu_fprintf); void dump_opcount_info(FILE *f, fprintf_function cpu_fprintf); diff --git a/include/exec/cpu-defs.h b/include/exec/cpu-defs.h index 98b9cff..a67f295 100644 --- a/include/exec/cpu-defs.h +++ b/include/exec/cpu-defs.h @@ -27,6 +27,7 @@ #include #include "qemu/osdep.h" #include "qemu/queue.h" +#include "qemu/range.h" #include "tcg-target.h" #ifndef CONFIG_USER_ONLY #include "exec/hwaddr.h" @@ -150,5 +151,16 @@ typedef struct CPUIOTLBEntry { #define CPU_COMMON \ /* soft mmu support */ \ CPU_COMMON_TLB \ + \ + /* Used by the atomic insn translation backend. */ \ + int ll_sc_context; \ + /* vCPU current exclusive addresses range. + * The address is set to EXCLUSIVE_RESET_ADDR if the vCPU is not. + * in the middle of a LL/SC. */ \ + struct Range excl_protected_range; \ + /* Used to carry the SC result but also to flag a normal (legacy) + * store access made by a stcond (see softmmu_template.h). */ \ + int excl_succeeded; \ + #endif diff --git a/softmmu_template.h b/softmmu_template.h index d42d89d..e4431e8 100644 --- a/softmmu_template.h +++ b/softmmu_template.h @@ -409,19 +409,53 @@ void helper_le_st_name(CPUArchState *env, target_ulong addr, DATA_TYPE val, tlb_addr = env->tlb_table[mmu_idx][index].addr_write; } - /* Handle an IO access. */ + /* Handle an IO access or exclusive access. */ if (unlikely(tlb_addr & ~TARGET_PAGE_MASK)) { - CPUIOTLBEntry *iotlbentry; - if ((addr & (DATA_SIZE - 1)) != 0) { - goto do_unaligned_access; + CPUIOTLBEntry *iotlbentry = &env->iotlb[mmu_idx][index]; + + if ((tlb_addr & ~TARGET_PAGE_MASK) == TLB_EXCL) { + /* The slow-path has been forced since we are writing to + * exclusive-protected memory. */ + hwaddr hw_addr = (iotlbentry->addr & TARGET_PAGE_MASK) + addr; + + /* The function lookup_and_reset_cpus_ll_addr could have reset the + * exclusive address. Fail the SC in this case. + * N.B.: Here excl_succeeded == 0 means that helper_le_st_name has + * not been called by a softmmu_llsc_template.h. */ + if(env->excl_succeeded) { + if (env->excl_protected_range.begin != hw_addr) { + /* The vCPU is SC-ing to an unprotected address. */ + env->excl_protected_range.begin = EXCLUSIVE_RESET_ADDR; + env->excl_succeeded = 0; + + return; + } + + cpu_physical_memory_set_excl_dirty(hw_addr, ENV_GET_CPU(env)->cpu_index); + } + + haddr = addr + env->tlb_table[mmu_idx][index].addend; + #if DATA_SIZE == 1 + glue(glue(st, SUFFIX), _p)((uint8_t *)haddr, val); + #else + glue(glue(st, SUFFIX), _le_p)((uint8_t *)haddr, val); + #endif + + lookup_and_reset_cpus_ll_addr(hw_addr, DATA_SIZE); + + return; + } else { + if ((addr & (DATA_SIZE - 1)) != 0) { + goto do_unaligned_access; + } + iotlbentry = &env->iotlb[mmu_idx][index]; + + /* ??? Note that the io helpers always read data in the target + byte ordering. We should push the LE/BE request down into io. */ + val = TGT_LE(val); + glue(io_write, SUFFIX)(env, iotlbentry, val, addr, retaddr); + return; } - iotlbentry = &env->iotlb[mmu_idx][index]; - - /* ??? Note that the io helpers always read data in the target - byte ordering. We should push the LE/BE request down into io. */ - val = TGT_LE(val); - glue(io_write, SUFFIX)(env, iotlbentry, val, addr, retaddr); - return; } /* Handle slow unaligned access (it spans two pages or IO). */ @@ -489,19 +523,53 @@ void helper_be_st_name(CPUArchState *env, target_ulong addr, DATA_TYPE val, tlb_addr = env->tlb_table[mmu_idx][index].addr_write; } - /* Handle an IO access. */ + /* Handle an IO access or exclusive access. */ if (unlikely(tlb_addr & ~TARGET_PAGE_MASK)) { - CPUIOTLBEntry *iotlbentry; - if ((addr & (DATA_SIZE - 1)) != 0) { - goto do_unaligned_access; + CPUIOTLBEntry *iotlbentry = &env->iotlb[mmu_idx][index]; + + if ((tlb_addr & ~TARGET_PAGE_MASK) == TLB_EXCL) { + /* The slow-path has been forced since we are writing to + * exclusive-protected memory. */ + hwaddr hw_addr = (iotlbentry->addr & TARGET_PAGE_MASK) + addr; + + /* The function lookup_and_reset_cpus_ll_addr could have reset the + * exclusive address. Fail the SC in this case. + * N.B.: Here excl_succeeded == 0 means that helper_le_st_name has + * not been called by a softmmu_llsc_template.h. */ + if(env->excl_succeeded) { + if (env->excl_protected_range.begin != hw_addr) { + /* The vCPU is SC-ing to an unprotected address. */ + env->excl_protected_range.begin = EXCLUSIVE_RESET_ADDR; + env->excl_succeeded = 0; + + return; + } + + cpu_physical_memory_set_excl_dirty(hw_addr, ENV_GET_CPU(env)->cpu_index); + } + + haddr = addr + env->tlb_table[mmu_idx][index].addend; + #if DATA_SIZE == 1 + glue(glue(st, SUFFIX), _p)((uint8_t *)haddr, val); + #else + glue(glue(st, SUFFIX), _le_p)((uint8_t *)haddr, val); + #endif + + lookup_and_reset_cpus_ll_addr(hw_addr, DATA_SIZE); + + return; + } else { + if ((addr & (DATA_SIZE - 1)) != 0) { + goto do_unaligned_access; + } + iotlbentry = &env->iotlb[mmu_idx][index]; + + /* ??? Note that the io helpers always read data in the target + byte ordering. We should push the LE/BE request down into io. */ + val = TGT_BE(val); + glue(io_write, SUFFIX)(env, iotlbentry, val, addr, retaddr); + return; } - iotlbentry = &env->iotlb[mmu_idx][index]; - - /* ??? Note that the io helpers always read data in the target - byte ordering. We should push the LE/BE request down into io. */ - val = TGT_BE(val); - glue(io_write, SUFFIX)(env, iotlbentry, val, addr, retaddr); - return; } /* Handle slow unaligned access (it spans two pages or IO). */