From patchwork Tue Apr 19 13:39:24 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alvise Rigo X-Patchwork-Id: 612153 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3qq5ms0PkMz9t8w for ; Tue, 19 Apr 2016 23:42:53 +1000 (AEST) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=virtualopensystems-com.20150623.gappssmtp.com header.i=@virtualopensystems-com.20150623.gappssmtp.com header.b=luMNWR25; dkim-atps=neutral Received: from localhost ([::1]:58158 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1asVvW-0001B7-Ty for incoming@patchwork.ozlabs.org; Tue, 19 Apr 2016 09:42:50 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57111) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1asVse-0003QR-HB for qemu-devel@nongnu.org; Tue, 19 Apr 2016 09:39:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1asVsc-0000hf-ED for qemu-devel@nongnu.org; Tue, 19 Apr 2016 09:39:52 -0400 Received: from mail-wm0-x244.google.com ([2a00:1450:400c:c09::244]:34535) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1asVsc-0000h9-4J for qemu-devel@nongnu.org; Tue, 19 Apr 2016 09:39:50 -0400 Received: by mail-wm0-x244.google.com with SMTP id n3so5310354wmn.1 for ; Tue, 19 Apr 2016 06:39:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtualopensystems-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=0MXjePy+3kMv7vW95upOD4iIb0wdsEuzW8eI4YMQt6I=; b=luMNWR25BI46wnX8FKMJs0DQ4AG0VBQQaE5lyFfNa3+XkUiMXj5pUEYHre/6kN7DZb COLS6q8wqiRgoO6nT0sAGhcfALzIKzvbFIBEOQaDIMBei9ANrq5vnI/d1++Cf3454owc 7wgaI+S1mgvxQRVwuM992f4/IrwkpBylVFuQnFsVJ4kqjp/O4FuTHzIiIm73rtOBAzHl x/7NSfTDUBsTGUYwCcglhzFwot/yi15F5AzpokSBnNvAd0jy6dui3u9xyo2OqwAA5xxj xI/TdM1P+y0nwiO/Hy7iYQbeqMcauTgbojePbrK8lk81QS2sVV9Wnw97TQVYNYF0XWF3 kdlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=0MXjePy+3kMv7vW95upOD4iIb0wdsEuzW8eI4YMQt6I=; b=MK/MQxCV4AwcjvoM/A+tHXsXcLJhwLM1M3BoYUt/M+U6VXn8i8fHclTy761aiS5zcH bv/ZMeDQJG+4+g4Pe+sURfPD5Gy7C64q4ynKSMA5nZZ+pc1Yey62rd/LbyBHnjQY2WuX 7trsQfeSke4zdhBIkAuCBr6GhOVT9TppVa+ZNlYWXN9Lz+Uj8XWgrOH1WKPFsC+IFdWd Oj8KIfGY9U2LyiGkKDkaw09YX3dT4MQEPQIc/SxDq3q6Uwxdj2g9gcygrqITnzvUN22n i9ta6EhD4cIa+/YEWCgNp6w5JLZLVHfW+jsSqNqN8sa175lcgU02WFO97Bef/Eqqkn+q D5cQ== X-Gm-Message-State: AOPr4FUTHajn3qmBk4xxrswIK3SCJ+9RUsRG55Jun6p8hjC+U3LC7LmO7YZJuthTqtR0Hw== X-Received: by 10.194.48.33 with SMTP id i1mr3771179wjn.53.1461073188332; Tue, 19 Apr 2016 06:39:48 -0700 (PDT) Received: from localhost.localdomain (LPuteaux-656-1-278-113.w80-15.abo.wanadoo.fr. [80.15.154.113]) by smtp.googlemail.com with ESMTPSA id c85sm2789756wmd.0.2016.04.19.06.39.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 19 Apr 2016 06:39:47 -0700 (PDT) From: Alvise Rigo To: qemu-devel@nongnu.org, mttcg@listserver.greensocs.com Date: Tue, 19 Apr 2016 15:39:24 +0200 Message-Id: <1461073171-22953-8-git-send-email-a.rigo@virtualopensystems.com> X-Mailer: git-send-email 2.8.0 In-Reply-To: <1461073171-22953-1-git-send-email-a.rigo@virtualopensystems.com> References: <1461073171-22953-1-git-send-email-a.rigo@virtualopensystems.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::244 Subject: [Qemu-devel] [RFC v8 07/14] softmmu: Add helpers for a new slowpath X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Crosthwaite , claudio.fontana@huawei.com, Alvise Rigo , serge.fdrv@gmail.com, pbonzini@redhat.com, jani.kokkonen@huawei.com, tech@virtualopensystems.com, alex.bennee@linaro.org, =?UTF-8?q?Andreas=20F=C3=A4rber?= , rth@twiddle.net Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" The new helpers rely on the legacy ones to perform the actual read/write. The LoadLink helper (helper_ldlink_name) prepares the way for the following StoreCond operation. It sets the linked address and the size of the access. The LoadLink helper also updates the TLB entry of the page involved in the LL/SC to all vCPUs by forcing a TLB flush, so that the following accesses made by all the vCPUs will follow the slow path. The StoreConditional helper (helper_stcond_name) returns 1 if the store has to fail due to a concurrent access to the same page by another vCPU. A 'concurrent access' can be a store made by *any* vCPU (although, some implementations allow stores made by the CPU that issued the LoadLink). For the time being we do not support exclusive accesses to MMIO memory. Suggested-by: Jani Kokkonen Suggested-by: Claudio Fontana Signed-off-by: Alvise Rigo --- cputlb.c | 4 ++ include/qom/cpu.h | 5 ++ qom/cpu.c | 2 + softmmu_llsc_template.h | 132 ++++++++++++++++++++++++++++++++++++++++++++++++ softmmu_template.h | 12 +++++ tcg/tcg.h | 31 ++++++++++++ 6 files changed, 186 insertions(+) create mode 100644 softmmu_llsc_template.h diff --git a/cputlb.c b/cputlb.c index f6fb161..58d6f03 100644 --- a/cputlb.c +++ b/cputlb.c @@ -29,6 +29,7 @@ #include "exec/memory-internal.h" #include "exec/ram_addr.h" #include "tcg/tcg.h" +#include "hw/hw.h" //#define DEBUG_TLB //#define DEBUG_TLB_CHECK @@ -476,6 +477,8 @@ tb_page_addr_t get_page_addr_code(CPUArchState *env1, target_ulong addr) #define MMUSUFFIX _mmu +/* Generates LoadLink/StoreConditional helpers in softmmu_template.h */ +#define GEN_EXCLUSIVE_HELPERS #define SHIFT 0 #include "softmmu_template.h" @@ -488,6 +491,7 @@ tb_page_addr_t get_page_addr_code(CPUArchState *env1, target_ulong addr) #define SHIFT 3 #include "softmmu_template.h" #undef MMUSUFFIX +#undef GEN_EXCLUSIVE_HELPERS #define MMUSUFFIX _cmmu #undef GETPC_ADJ diff --git a/include/qom/cpu.h b/include/qom/cpu.h index 21f10eb..014851e 100644 --- a/include/qom/cpu.h +++ b/include/qom/cpu.h @@ -356,10 +356,15 @@ struct CPUState { */ bool throttle_thread_scheduled; + /* Used by the atomic insn translation backend. */ + bool ll_sc_context; /* vCPU's exclusive addresses range. * The address is set to EXCLUSIVE_RESET_ADDR if the vCPU is not * in the middle of a LL/SC. */ struct Range excl_protected_range; + /* Used to carry the SC result but also to flag a normal store access made + * by a stcond (see softmmu_template.h). */ + bool excl_succeeded; /* Note that this is accessed at the start of every TB via a negative offset from AREG0. Leave this field at the end so as to make the diff --git a/qom/cpu.c b/qom/cpu.c index 309d487..3280735 100644 --- a/qom/cpu.c +++ b/qom/cpu.c @@ -224,6 +224,8 @@ static bool cpu_common_valid_excl_access(CPUState *cpu, hwaddr addr, hwaddr size static void cpu_common_reset_excl_context(CPUState *cpu) { cpu->excl_protected_range.begin = EXCLUSIVE_RESET_ADDR; + cpu->ll_sc_context = false; + cpu->excl_succeeded = false; } void cpu_dump_state(CPUState *cpu, FILE *f, fprintf_function cpu_fprintf, diff --git a/softmmu_llsc_template.h b/softmmu_llsc_template.h new file mode 100644 index 0000000..ca2ac95 --- /dev/null +++ b/softmmu_llsc_template.h @@ -0,0 +1,132 @@ +/* + * Software MMU support (esclusive load/store operations) + * + * Generate helpers used by TCG for qemu_ldlink/stcond ops. + * + * Included from softmmu_template.h only. + * + * Copyright (c) 2015 Virtual Open Systems + * + * Authors: + * Alvise Rigo + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, see . + */ + +/* This template does not generate together the le and be version, but only one + * of the two depending on whether BIGENDIAN_EXCLUSIVE_HELPERS has been set. + * The same nomenclature as softmmu_template.h is used for the exclusive + * helpers. */ + +#ifdef BIGENDIAN_EXCLUSIVE_HELPERS + +#define helper_ldlink_name glue(glue(helper_be_ldlink, USUFFIX), MMUSUFFIX) +#define helper_stcond_name glue(glue(helper_be_stcond, SUFFIX), MMUSUFFIX) +#define helper_ld glue(glue(helper_be_ld, USUFFIX), MMUSUFFIX) +#define helper_st glue(glue(helper_be_st, SUFFIX), MMUSUFFIX) + +#else /* LE helpers + 8bit helpers (generated only once for both LE end BE) */ + +#if DATA_SIZE > 1 +#define helper_ldlink_name glue(glue(helper_le_ldlink, USUFFIX), MMUSUFFIX) +#define helper_stcond_name glue(glue(helper_le_stcond, SUFFIX), MMUSUFFIX) +#define helper_ld glue(glue(helper_le_ld, USUFFIX), MMUSUFFIX) +#define helper_st glue(glue(helper_le_st, SUFFIX), MMUSUFFIX) +#else /* DATA_SIZE <= 1 */ +#define helper_ldlink_name glue(glue(helper_ret_ldlink, USUFFIX), MMUSUFFIX) +#define helper_stcond_name glue(glue(helper_ret_stcond, SUFFIX), MMUSUFFIX) +#define helper_ld glue(glue(helper_ret_ld, USUFFIX), MMUSUFFIX) +#define helper_st glue(glue(helper_ret_st, SUFFIX), MMUSUFFIX) +#endif + +#endif + +WORD_TYPE helper_ldlink_name(CPUArchState *env, target_ulong addr, + TCGMemOpIdx oi, uintptr_t retaddr) +{ + WORD_TYPE ret; + int index; + CPUState *this_cpu = ENV_GET_CPU(env); + CPUClass *cc = CPU_GET_CLASS(this_cpu); + hwaddr hw_addr; + unsigned mmu_idx = get_mmuidx(oi); + + /* Use the proper load helper from cpu_ldst.h */ + ret = helper_ld(env, addr, oi, retaddr); + + index = (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); + + /* hw_addr = hwaddr of the page (i.e. section->mr->ram_addr + xlat) + * plus the offset (i.e. addr & ~TARGET_PAGE_MASK) */ + hw_addr = (env->iotlb[mmu_idx][index].addr & TARGET_PAGE_MASK) + addr; + if (likely(!(env->tlb_table[mmu_idx][index].addr_read & TLB_MMIO))) { + /* If all the vCPUs have the EXCL bit set for this page there is no need + * to request any flush. */ + if (!cpu_physical_memory_is_excl(hw_addr)) { + CPUState *cpu; + + cpu_physical_memory_set_excl(hw_addr); + CPU_FOREACH(cpu) { + if (this_cpu != cpu) { + tlb_flush(cpu, 1); + } + } + } + } else { + hw_error("EXCL accesses to MMIO regions not supported yet."); + } + + cc->cpu_set_excl_protected_range(this_cpu, hw_addr, DATA_SIZE); + + /* For this vCPU, just update the TLB entry, no need to flush. */ + env->tlb_table[mmu_idx][index].addr_write |= TLB_EXCL; + + /* From now on we are in LL/SC context */ + this_cpu->ll_sc_context = true; + + return ret; +} + +WORD_TYPE helper_stcond_name(CPUArchState *env, target_ulong addr, + DATA_TYPE val, TCGMemOpIdx oi, + uintptr_t retaddr) +{ + WORD_TYPE ret = 0; + CPUState *cpu = ENV_GET_CPU(env); + CPUClass *cc = CPU_GET_CLASS(cpu); + + if (!cpu->ll_sc_context) { + ret = 1; + } else { + /* We set it preventively to true to distinguish the following legacy + * access as one made by the store conditional wrapper. If the store + * conditional does not succeed, the value will be set to 0.*/ + cpu->excl_succeeded = true; + helper_st(env, addr, val, oi, retaddr); + + if (!cpu->excl_succeeded) { + ret = 1; + } + } + + /* Unset LL/SC context */ + cc->cpu_reset_excl_context(cpu); + + return ret; +} + +#undef helper_ldlink_name +#undef helper_stcond_name +#undef helper_ld +#undef helper_st diff --git a/softmmu_template.h b/softmmu_template.h index ea6a0fb..ede1240 100644 --- a/softmmu_template.h +++ b/softmmu_template.h @@ -565,6 +565,18 @@ void probe_write(CPUArchState *env, target_ulong addr, int mmu_idx, #endif #endif /* !defined(SOFTMMU_CODE_ACCESS) */ +#ifdef GEN_EXCLUSIVE_HELPERS + +#if DATA_SIZE > 1 /* The 8-bit helpers are generate along with LE helpers */ +#define BIGENDIAN_EXCLUSIVE_HELPERS +#include "softmmu_llsc_template.h" +#undef BIGENDIAN_EXCLUSIVE_HELPERS +#endif + +#include "softmmu_llsc_template.h" + +#endif /* !defined(GEN_EXCLUSIVE_HELPERS) */ + #undef READ_ACCESS_TYPE #undef SHIFT #undef DATA_TYPE diff --git a/tcg/tcg.h b/tcg/tcg.h index a696922..3e050a4 100644 --- a/tcg/tcg.h +++ b/tcg/tcg.h @@ -968,6 +968,21 @@ tcg_target_ulong helper_be_ldul_mmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr); uint64_t helper_be_ldq_mmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr); +/* Exclusive variants */ +tcg_target_ulong helper_ret_ldlinkub_mmu(CPUArchState *env, target_ulong addr, + TCGMemOpIdx oi, uintptr_t retaddr); +tcg_target_ulong helper_le_ldlinkuw_mmu(CPUArchState *env, target_ulong addr, + TCGMemOpIdx oi, uintptr_t retaddr); +tcg_target_ulong helper_le_ldlinkul_mmu(CPUArchState *env, target_ulong addr, + TCGMemOpIdx oi, uintptr_t retaddr); +uint64_t helper_le_ldlinkq_mmu(CPUArchState *env, target_ulong addr, + TCGMemOpIdx oi, uintptr_t retaddr); +tcg_target_ulong helper_be_ldlinkuw_mmu(CPUArchState *env, target_ulong addr, + TCGMemOpIdx oi, uintptr_t retaddr); +tcg_target_ulong helper_be_ldlinkul_mmu(CPUArchState *env, target_ulong addr, + TCGMemOpIdx oi, uintptr_t retaddr); +uint64_t helper_be_ldlinkq_mmu(CPUArchState *env, target_ulong addr, + TCGMemOpIdx oi, uintptr_t retaddr); /* Value sign-extended to tcg register size. */ tcg_target_ulong helper_ret_ldsb_mmu(CPUArchState *env, target_ulong addr, @@ -1010,6 +1025,22 @@ uint32_t helper_be_ldl_cmmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr); uint64_t helper_be_ldq_cmmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr); +/* Exclusive variants */ +tcg_target_ulong helper_ret_stcondb_mmu(CPUArchState *env, target_ulong addr, + uint8_t val, TCGMemOpIdx oi, uintptr_t retaddr); +tcg_target_ulong helper_le_stcondw_mmu(CPUArchState *env, target_ulong addr, + uint16_t val, TCGMemOpIdx oi, uintptr_t retaddr); +tcg_target_ulong helper_le_stcondl_mmu(CPUArchState *env, target_ulong addr, + uint32_t val, TCGMemOpIdx oi, uintptr_t retaddr); +uint64_t helper_le_stcondq_mmu(CPUArchState *env, target_ulong addr, + uint64_t val, TCGMemOpIdx oi, uintptr_t retaddr); +tcg_target_ulong helper_be_stcondw_mmu(CPUArchState *env, target_ulong addr, + uint16_t val, TCGMemOpIdx oi, uintptr_t retaddr); +tcg_target_ulong helper_be_stcondl_mmu(CPUArchState *env, target_ulong addr, + uint32_t val, TCGMemOpIdx oi, uintptr_t retaddr); +uint64_t helper_be_stcondq_mmu(CPUArchState *env, target_ulong addr, + uint64_t val, TCGMemOpIdx oi, uintptr_t retaddr); + /* Temporary aliases until backends are converted. */ #ifdef TARGET_WORDS_BIGENDIAN