From patchwork Fri Aug 7 17:03:12 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alvise Rigo X-Patchwork-Id: 505203 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id CD7FC140285 for ; Sat, 8 Aug 2015 03:05:16 +1000 (AEST) Received: from localhost ([::1]:50019 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZNl51-0002X6-0m for incoming@patchwork.ozlabs.org; Fri, 07 Aug 2015 13:05:15 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:36261) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZNl0d-0002bH-Md for qemu-devel@nongnu.org; Fri, 07 Aug 2015 13:00:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ZNl0Z-0004FR-TE for qemu-devel@nongnu.org; Fri, 07 Aug 2015 13:00:43 -0400 Received: from mail-wi0-f177.google.com ([209.85.212.177]:35458) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ZNl0Y-0004Ev-Sz for qemu-devel@nongnu.org; Fri, 07 Aug 2015 13:00:39 -0400 Received: by wibxm9 with SMTP id xm9so68833853wib.0 for ; Fri, 07 Aug 2015 10:00:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=gA6gXAh8qWXMseAgSkNT/MPeCLS8A7hsEN1hL7zEYaI=; b=ReLHvRriaco1HT4QqlDjWPo4DBvBCPyol26dK9dX4q3KdyAI0YS5sr4LfwiPA/E9YK wPWtz8i4A4YRiDTJrLPJx1NlsuWfv/gGL/RXHLQebbkhfDgRrl19Np711zaO5vFC35l/ G1TdxNgi3KIJGkWOixnqvKUoN+fs4GRDv9CCjDMMW0/HK2zmXUj+n6TH3RB+qJ69XTo/ vSzD07HDseI3fhUPnhSqyz9j/v8btl5L+cUjKhOtaxVOZxInfAzBCAbVKBiYqW0V90e6 4xuG4eWz06DFwGj15SQzb33nUB7lqNnMfTv8naNs/vQtWOyMnrNKAT/rRsbgV80lKX5+ Pf7w== X-Gm-Message-State: ALoCoQl428oDK+OV7D1QdrRme1g8SaeX5lKt87Mxh5G+eGkJyQNgtrMsf3BJfepb7NMq1hrOpJGo X-Received: by 10.180.98.134 with SMTP id ei6mr7807236wib.49.1438966838272; Fri, 07 Aug 2015 10:00:38 -0700 (PDT) Received: from linarch.home (LPuteaux-656-1-278-113.w80-15.abo.wanadoo.fr. [80.15.154.113]) by smtp.googlemail.com with ESMTPSA id c7sm15490379wjb.19.2015.08.07.10.00.37 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 07 Aug 2015 10:00:37 -0700 (PDT) From: Alvise Rigo To: qemu-devel@nongnu.org, mttcg@listserver.greensocs.com Date: Fri, 7 Aug 2015 19:03:12 +0200 Message-Id: <1438966995-5913-7-git-send-email-a.rigo@virtualopensystems.com> X-Mailer: git-send-email 2.5.0 In-Reply-To: <1438966995-5913-1-git-send-email-a.rigo@virtualopensystems.com> References: <1438966995-5913-1-git-send-email-a.rigo@virtualopensystems.com> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.85.212.177 Cc: alex.bennee@linaro.org, jani.kokkonen@huawei.com, tech@virtualopensystems.com, claudio.fontana@huawei.com, pbonzini@redhat.com Subject: [Qemu-devel] [RFC v4 6/9] tcg-i386: Implement excl variants of qemu_{ld, st} X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Implement exclusive variants of qemu_{ld,st}_{i32,i64} for tcg-i386. The lookup for the proper memory helper has been rewritten to take into account the new exclusive helpers. Suggested-by: Jani Kokkonen Suggested-by: Claudio Fontana Signed-off-by: Alvise Rigo --- tcg/i386/tcg-target.c | 148 +++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 129 insertions(+), 19 deletions(-) diff --git a/tcg/i386/tcg-target.c b/tcg/i386/tcg-target.c index ff4d9cf..011907c 100644 --- a/tcg/i386/tcg-target.c +++ b/tcg/i386/tcg-target.c @@ -1137,6 +1137,28 @@ static void * const qemu_ld_helpers[16] = { [MO_BEQ] = helper_be_ldq_mmu, }; +/* LoadLink helpers, only unsigned. Use the macro below to access them. */ +static void * const qemu_ldex_helpers[16] = { + [MO_UB] = helper_ret_ldlinkub_mmu, + + [MO_LEUW] = helper_le_ldlinkuw_mmu, + [MO_LEUL] = helper_le_ldlinkul_mmu, + [MO_LEQ] = helper_le_ldlinkq_mmu, + + [MO_BEUW] = helper_be_ldlinkuw_mmu, + [MO_BEUL] = helper_be_ldlinkul_mmu, + [MO_BEQ] = helper_be_ldlinkq_mmu, +}; + +static inline tcg_insn_unit *ld_helper(TCGMemOp opc) +{ + if (opc & MO_EXCL) { + return qemu_ldex_helpers[((int)opc - MO_EXCL) & (MO_BSWAP | MO_SSIZE)]; + } + + return qemu_ld_helpers[opc & ~MO_SIGN]; +} + /* helper signature: helper_ret_st_mmu(CPUState *env, target_ulong addr, * uintxx_t val, int mmu_idx, uintptr_t ra) */ @@ -1150,6 +1172,26 @@ static void * const qemu_st_helpers[16] = { [MO_BEQ] = helper_be_stq_mmu, }; +/* StoreConditional helpers. Use the macro below to access them. */ +static void * const qemu_stex_helpers[16] = { + [MO_UB] = helper_ret_stcondb_mmu, + [MO_LEUW] = helper_le_stcondw_mmu, + [MO_LEUL] = helper_le_stcondl_mmu, + [MO_LEQ] = helper_le_stcondq_mmu, + [MO_BEUW] = helper_be_stcondw_mmu, + [MO_BEUL] = helper_be_stcondl_mmu, + [MO_BEQ] = helper_be_stcondq_mmu, +}; + +static inline tcg_insn_unit *st_helper(TCGMemOp opc) +{ + if (opc & MO_EXCL) { + return qemu_stex_helpers[((int)opc - MO_EXCL) & (MO_BSWAP | MO_SSIZE)]; + } + + return qemu_st_helpers[opc]; +} + /* Perform the TLB load and compare. Inputs: @@ -1245,6 +1287,7 @@ static inline void tcg_out_tlb_load(TCGContext *s, TCGReg addrlo, TCGReg addrhi, * for a load or store, so that we can later generate the correct helper code */ static void add_qemu_ldst_label(TCGContext *s, bool is_ld, TCGMemOpIdx oi, + TCGReg llsc_success, TCGReg datalo, TCGReg datahi, TCGReg addrlo, TCGReg addrhi, tcg_insn_unit *raddr, @@ -1253,6 +1296,7 @@ static void add_qemu_ldst_label(TCGContext *s, bool is_ld, TCGMemOpIdx oi, TCGLabelQemuLdst *label = new_ldst_label(s); label->is_ld = is_ld; + label->llsc_success = llsc_success; label->oi = oi; label->datalo_reg = datalo; label->datahi_reg = datahi; @@ -1307,7 +1351,7 @@ static void tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *l) (uintptr_t)l->raddr); } - tcg_out_call(s, qemu_ld_helpers[opc & (MO_BSWAP | MO_SIZE)]); + tcg_out_call(s, ld_helper(opc)); data_reg = l->datalo_reg; switch (opc & MO_SSIZE) { @@ -1411,9 +1455,16 @@ static void tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *l) } } - /* "Tail call" to the helper, with the return address back inline. */ - tcg_out_push(s, retaddr); - tcg_out_jmp(s, qemu_st_helpers[opc & (MO_BSWAP | MO_SIZE)]); + if (opc & MO_EXCL) { + tcg_out_call(s, st_helper(opc)); + /* Save the output of the StoreConditional */ + tcg_out_mov(s, TCG_TYPE_I32, l->llsc_success, TCG_REG_EAX); + tcg_out_jmp(s, l->raddr); + } else { + /* "Tail call" to the helper, with the return address back inline. */ + tcg_out_push(s, retaddr); + tcg_out_jmp(s, st_helper(opc)); + } } #elif defined(__x86_64__) && defined(__linux__) # include @@ -1549,14 +1600,34 @@ static void tcg_out_qemu_ld(TCGContext *s, const TCGArg *args, bool is64) mem_index = get_mmuidx(oi); s_bits = opc & MO_SIZE; - tcg_out_tlb_load(s, addrlo, addrhi, mem_index, s_bits, - label_ptr, offsetof(CPUTLBEntry, addr_read)); + if (opc & MO_EXCL) { + TCGType t = ((TCG_TARGET_REG_BITS == 64) && (TARGET_LONG_BITS == 64)) ? + TCG_TYPE_I64 : TCG_TYPE_I32; + /* The JMP address will be patched afterwards, + * in tcg_out_qemu_ld_slow_path (two times when + * TARGET_LONG_BITS > TCG_TARGET_REG_BITS). */ + tcg_out_mov(s, t, TCG_REG_L1, addrlo); + + if (TARGET_LONG_BITS > TCG_TARGET_REG_BITS) { + /* Store the second part of the address. */ + tcg_out_mov(s, t, TCG_REG_L0, addrhi); + /* We add 4 to include the jmp that follows. */ + label_ptr[1] = s->code_ptr + 4; + } - /* TLB Hit. */ - tcg_out_qemu_ld_direct(s, datalo, datahi, TCG_REG_L1, 0, 0, opc); + tcg_out_opc(s, OPC_JMP_long, 0, 0, 0); + label_ptr[0] = s->code_ptr; + s->code_ptr += 4; + } else { + tcg_out_tlb_load(s, addrlo, addrhi, mem_index, s_bits, + label_ptr, offsetof(CPUTLBEntry, addr_read)); + + /* TLB Hit. */ + tcg_out_qemu_ld_direct(s, datalo, datahi, TCG_REG_L1, 0, 0, opc); + } /* Record the current context of a load into ldst label */ - add_qemu_ldst_label(s, true, oi, datalo, datahi, addrlo, addrhi, + add_qemu_ldst_label(s, true, oi, 0, datalo, datahi, addrlo, addrhi, s->code_ptr, label_ptr); #else { @@ -1659,9 +1730,10 @@ static void tcg_out_qemu_st_direct(TCGContext *s, TCGReg datalo, TCGReg datahi, } } -static void tcg_out_qemu_st(TCGContext *s, const TCGArg *args, bool is64) +static void tcg_out_qemu_st(TCGContext *s, const TCGArg *args, bool is64, + bool isStoreCond) { - TCGReg datalo, datahi, addrlo; + TCGReg datalo, datahi, addrlo, llsc_success; TCGReg addrhi __attribute__((unused)); TCGMemOpIdx oi; TCGMemOp opc; @@ -1671,6 +1743,9 @@ static void tcg_out_qemu_st(TCGContext *s, const TCGArg *args, bool is64) tcg_insn_unit *label_ptr[2]; #endif + /* The stcond variant has one more param */ + llsc_success = (isStoreCond ? *args++ : 0); + datalo = *args++; datahi = (TCG_TARGET_REG_BITS == 32 && is64 ? *args++ : 0); addrlo = *args++; @@ -1682,15 +1757,35 @@ static void tcg_out_qemu_st(TCGContext *s, const TCGArg *args, bool is64) mem_index = get_mmuidx(oi); s_bits = opc & MO_SIZE; - tcg_out_tlb_load(s, addrlo, addrhi, mem_index, s_bits, - label_ptr, offsetof(CPUTLBEntry, addr_write)); + if (opc & MO_EXCL) { + TCGType t = ((TCG_TARGET_REG_BITS == 64) && (TARGET_LONG_BITS == 64)) ? + TCG_TYPE_I64 : TCG_TYPE_I32; + /* The JMP address will be filled afterwards, + * in tcg_out_qemu_ld_slow_path (two times when + * TARGET_LONG_BITS > TCG_TARGET_REG_BITS). */ + tcg_out_mov(s, t, TCG_REG_L1, addrlo); + + if (TARGET_LONG_BITS > TCG_TARGET_REG_BITS) { + /* Store the second part of the address. */ + tcg_out_mov(s, t, TCG_REG_L0, addrhi); + /* We add 4 to include the jmp that follows. */ + label_ptr[1] = s->code_ptr + 4; + } - /* TLB Hit. */ - tcg_out_qemu_st_direct(s, datalo, datahi, TCG_REG_L1, 0, 0, opc); + tcg_out_opc(s, OPC_JMP_long, 0, 0, 0); + label_ptr[0] = s->code_ptr; + s->code_ptr += 4; + } else { + tcg_out_tlb_load(s, addrlo, addrhi, mem_index, s_bits, + label_ptr, offsetof(CPUTLBEntry, addr_write)); + + /* TLB Hit. */ + tcg_out_qemu_st_direct(s, datalo, datahi, TCG_REG_L1, 0, 0, opc); + } /* Record the current context of a store into ldst label */ - add_qemu_ldst_label(s, false, oi, datalo, datahi, addrlo, addrhi, - s->code_ptr, label_ptr); + add_qemu_ldst_label(s, false, oi, llsc_success, datalo, datahi, addrlo, + addrhi, s->code_ptr, label_ptr); #else { int32_t offset = GUEST_BASE; @@ -1957,10 +2052,16 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode opc, tcg_out_qemu_ld(s, args, 1); break; case INDEX_op_qemu_st_i32: - tcg_out_qemu_st(s, args, 0); + tcg_out_qemu_st(s, args, 0, 0); + break; + case INDEX_op_qemu_stcond_i32: + tcg_out_qemu_st(s, args, 0, 1); break; case INDEX_op_qemu_st_i64: - tcg_out_qemu_st(s, args, 1); + tcg_out_qemu_st(s, args, 1, 0); + break; + case INDEX_op_qemu_stcond_i64: + tcg_out_qemu_st(s, args, 1, 1); break; OP_32_64(mulu2): @@ -2182,19 +2283,28 @@ static const TCGTargetOpDef x86_op_defs[] = { #if TCG_TARGET_REG_BITS == 64 { INDEX_op_qemu_ld_i32, { "r", "L" } }, + { INDEX_op_qemu_ldlink_i32, { "r", "L" } }, { INDEX_op_qemu_st_i32, { "L", "L" } }, + { INDEX_op_qemu_stcond_i32, { "r", "L", "L" } }, { INDEX_op_qemu_ld_i64, { "r", "L" } }, { INDEX_op_qemu_st_i64, { "L", "L" } }, + { INDEX_op_qemu_stcond_i64, { "r", "L", "L" } }, #elif TARGET_LONG_BITS <= TCG_TARGET_REG_BITS { INDEX_op_qemu_ld_i32, { "r", "L" } }, + { INDEX_op_qemu_ldlink_i32, { "r", "L" } }, { INDEX_op_qemu_st_i32, { "L", "L" } }, + { INDEX_op_qemu_stcond_i32, { "r", "L", "L" } }, { INDEX_op_qemu_ld_i64, { "r", "r", "L" } }, { INDEX_op_qemu_st_i64, { "L", "L", "L" } }, + { INDEX_op_qemu_stcond_i64, { "r", "L", "L", "L" } }, #else { INDEX_op_qemu_ld_i32, { "r", "L", "L" } }, + { INDEX_op_qemu_ldlink_i32, { "r", "L", "L" } }, { INDEX_op_qemu_st_i32, { "L", "L", "L" } }, + { INDEX_op_qemu_stcond_i32, { "r", "L", "L", "L" } }, { INDEX_op_qemu_ld_i64, { "r", "r", "L", "L" } }, { INDEX_op_qemu_st_i64, { "L", "L", "L", "L" } }, + { INDEX_op_qemu_stcond_i64, { "r", "L", "L", "L", "L" } }, #endif { -1 }, };