From patchwork Mon Aug 28 18:55:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 1826871 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.a=rsa-sha256 header.s=google header.b=V+4+WRs+; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RZKYB6HG9z1yhW for ; Tue, 29 Aug 2023 04:57:02 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qahP8-0000ql-If; Mon, 28 Aug 2023 14:56:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qahP3-0000oV-BB for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:55:57 -0400 Received: from mail-pj1-x1032.google.com ([2607:f8b0:4864:20::1032]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qahP0-0002QW-R8 for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:55:57 -0400 Received: by mail-pj1-x1032.google.com with SMTP id 98e67ed59e1d1-26d1a17ce06so1725245a91.0 for ; Mon, 28 Aug 2023 11:55:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1693248953; x=1693853753; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=03eKjHBSqhHdZ2C0Bhm1xZE8o9z6xsoYmMLEvVPH1DU=; b=V+4+WRs+wA91WtdmjuCmZv3m/bypY/4YkVMNzRqFJ0UWrTlNPYFZDRyvJUfuvpKsBT OgYD76e5SRD9GHYXTBB7vvGE4ILru+OFnoBy+OW/7iEvOVRx0Z1AJJGMJ+lhFWgv4Ndk 7qhEpVoNb4R9eh7NIZrnqKJblISeg/hsjdgSFmBAfrRbmsRMwRaAfVMkkLZ2v4AITgW4 5HnK2DsdE2psAAPBqRuwqKGRwUH0HEHmoOkSBrDweiqB9XaPTgC2yPhF+5AgoEq0Tun8 83DzEgyFD5TOSVGxcDRG6UNRN5UQs0qA+gsnH1eeqcstD9xltohyhq7NaGUmdhieA5mT dDnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693248953; x=1693853753; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=03eKjHBSqhHdZ2C0Bhm1xZE8o9z6xsoYmMLEvVPH1DU=; b=P3nMveWNkQGdXqzmVSw7SHQstd5pA/wNi1hKh7A+eI/FqTgBh33S9VWeYuU7UaGoCB qnQfEnWwoR2G3ucmirFDRzO/bdiK+gmrrwojBqLdzfz+EpFCuxpMj8ZDuvubiCdsThke 1ne1Sv8Bs9GiqnQm7QSWy8hdYxFLeN2clvNHzSKqR8jz2fYHJifjk8S1w8e2DzxfMRD5 WytNffWFJ1qrnsLRg6OLfSqLZSGJOw7Y1jvPRiMqAujqKKRNUgkKNaJIaZWN61HFGzKY ONraQCew9r0p0/oOyWbhow3FZudGJtPZvxw5mbfjRycW0brCnQb2P+jpYMeF0JdYOBPZ 9Cog== X-Gm-Message-State: AOJu0Yz/YJVQ1XguCqukDjo74mlZccv/iQ+l+lmHTEzvzWQxV2u1c/Qo YaI7b2vndJ96gGMqksII88EU844YJ5QGbDFh9Us= X-Google-Smtp-Source: AGHT+IEVOGjGCZAikiv2ZkLsXqzCZBOCeQXqKKjv7B7QXOwqeUfRjepT7HsuhqsB2stJkFqkbpIHkA== X-Received: by 2002:a17:90a:a389:b0:268:4314:2dc6 with SMTP id x9-20020a17090aa38900b0026843142dc6mr20076489pjp.37.1693248953198; Mon, 28 Aug 2023 11:55:53 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id 4-20020a17090a190400b002680dfd368dsm7932835pjg.51.2023.08.28.11.55.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Aug 2023 11:55:52 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: alex.bennee@linaro.org Subject: [PATCH 01/10] accel/tcg: Simplify tlb_plugin_lookup Date: Mon, 28 Aug 2023 11:55:41 -0700 Message-Id: <20230828185550.573653-2-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230828185550.573653-1-richard.henderson@linaro.org> References: <20230828185550.573653-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1032; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1032.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Now that we defer address space update and tlb_flush until the next async_run_on_cpu, the plugin run at the end of the instruction no longer has to contend with a flushed tlb. Therefore, delete SavedIOTLB entirely. Properly return false from tlb_plugin_lookup when we do not have a tlb match. Fixes a bug in which SavedIOTLB had stale data, because there were multiple i/o accesses within a single insn. Signed-off-by: Richard Henderson --- include/hw/core/cpu.h | 13 ------- include/qemu/typedefs.h | 1 - accel/tcg/cputlb.c | 79 ++++++++++++----------------------------- 3 files changed, 23 insertions(+), 70 deletions(-) diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h index fdcbe87352..20e4f64d72 100644 --- a/include/hw/core/cpu.h +++ b/include/hw/core/cpu.h @@ -227,17 +227,6 @@ struct CPUWatchpoint { QTAILQ_ENTRY(CPUWatchpoint) entry; }; -#ifdef CONFIG_PLUGIN -/* - * For plugins we sometime need to save the resolved iotlb data before - * the memory regions get moved around by io_writex. - */ -typedef struct SavedIOTLB { - MemoryRegionSection *section; - hwaddr mr_offset; -} SavedIOTLB; -#endif - struct KVMState; struct kvm_run; @@ -409,8 +398,6 @@ struct CPUState { #ifdef CONFIG_PLUGIN GArray *plugin_mem_cbs; - /* saved iotlb data from io_writex */ - SavedIOTLB saved_iotlb; #endif /* TODO Move common fields from CPUArchState here. */ diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h index 834b0e47a0..5abdbc3874 100644 --- a/include/qemu/typedefs.h +++ b/include/qemu/typedefs.h @@ -129,7 +129,6 @@ typedef struct QString QString; typedef struct RAMBlock RAMBlock; typedef struct Range Range; typedef struct ReservedRegion ReservedRegion; -typedef struct SavedIOTLB SavedIOTLB; typedef struct SHPCDevice SHPCDevice; typedef struct SSIBus SSIBus; typedef struct TCGHelperInfo TCGHelperInfo; diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index c643d66190..fcf6ccebff 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1364,21 +1364,6 @@ static inline void cpu_transaction_failed(CPUState *cpu, hwaddr physaddr, } } -/* - * Save a potentially trashed CPUTLBEntryFull for later lookup by plugin. - * This is read by tlb_plugin_lookup if the fulltlb entry doesn't match - * because of the side effect of io_writex changing memory layout. - */ -static void save_iotlb_data(CPUState *cs, MemoryRegionSection *section, - hwaddr mr_offset) -{ -#ifdef CONFIG_PLUGIN - SavedIOTLB *saved = &cs->saved_iotlb; - saved->section = section; - saved->mr_offset = mr_offset; -#endif -} - static uint64_t io_readx(CPUArchState *env, CPUTLBEntryFull *full, int mmu_idx, vaddr addr, uintptr_t retaddr, MMUAccessType access_type, MemOp op) @@ -1398,12 +1383,6 @@ static uint64_t io_readx(CPUArchState *env, CPUTLBEntryFull *full, cpu_io_recompile(cpu, retaddr); } - /* - * The memory_region_dispatch may trigger a flush/resize - * so for plugins we save the iotlb_data just in case. - */ - save_iotlb_data(cpu, section, mr_offset); - { QEMU_IOTHREAD_LOCK_GUARD(); r = memory_region_dispatch_read(mr, mr_offset, &val, op, full->attrs); @@ -1438,12 +1417,6 @@ static void io_writex(CPUArchState *env, CPUTLBEntryFull *full, } cpu->mem_io_pc = retaddr; - /* - * The memory_region_dispatch may trigger a flush/resize - * so for plugins we save the iotlb_data just in case. - */ - save_iotlb_data(cpu, section, mr_offset); - { QEMU_IOTHREAD_LOCK_GUARD(); r = memory_region_dispatch_write(mr, mr_offset, val, op, full->attrs); @@ -1726,45 +1699,39 @@ tb_page_addr_t get_page_addr_code_hostp(CPUArchState *env, vaddr addr, * in the softmmu lookup code (or helper). We don't handle re-fills or * checking the victim table. This is purely informational. * - * This almost never fails as the memory access being instrumented - * should have just filled the TLB. The one corner case is io_writex - * which can cause TLB flushes and potential resizing of the TLBs - * losing the information we need. In those cases we need to recover - * data from a copy of the CPUTLBEntryFull. As long as this always occurs - * from the same thread (which a mem callback will be) this is safe. + * The one corner case is i/o write, which can cause changes to the + * address space. Those changes, and the corresponding tlb flush, + * should be delayed until the next TB, so even then this ought not fail. + * But check, Just in Case. */ - bool tlb_plugin_lookup(CPUState *cpu, vaddr addr, int mmu_idx, bool is_store, struct qemu_plugin_hwaddr *data) { CPUArchState *env = cpu->env_ptr; CPUTLBEntry *tlbe = tlb_entry(env, mmu_idx, addr); uintptr_t index = tlb_index(env, mmu_idx, addr); - uint64_t tlb_addr = is_store ? tlb_addr_write(tlbe) : tlbe->addr_read; + MMUAccessType access_type = is_store ? MMU_DATA_STORE : MMU_DATA_LOAD; + uint64_t tlb_addr = tlb_read_idx(tlbe, access_type); - if (likely(tlb_hit(tlb_addr, addr))) { - /* We must have an iotlb entry for MMIO */ - if (tlb_addr & TLB_MMIO) { - CPUTLBEntryFull *full; - full = &env_tlb(env)->d[mmu_idx].fulltlb[index]; - data->is_io = true; - data->v.io.section = - iotlb_to_section(cpu, full->xlat_section, full->attrs); - data->v.io.offset = (full->xlat_section & TARGET_PAGE_MASK) + addr; - } else { - data->is_io = false; - data->v.ram.hostaddr = (void *)((uintptr_t)addr + tlbe->addend); - } - return true; - } else { - SavedIOTLB *saved = &cpu->saved_iotlb; - data->is_io = true; - data->v.io.section = saved->section; - data->v.io.offset = saved->mr_offset; - return true; + if (unlikely(!tlb_hit(tlb_addr, addr))) { + return false; } -} + /* We must have an iotlb entry for MMIO */ + if (tlb_addr & TLB_MMIO) { + CPUTLBEntryFull *full = &env_tlb(env)->d[mmu_idx].fulltlb[index]; + hwaddr xlat = full->xlat_section; + + data->is_io = true; + data->v.io.offset = (xlat & TARGET_PAGE_MASK) + addr; + data->v.io.section = + iotlb_to_section(cpu, xlat & ~TARGET_PAGE_MASK, full->attrs); + } else { + data->is_io = false; + data->v.ram.hostaddr = (void *)((uintptr_t)addr + tlbe->addend); + } + return true; +} #endif /* From patchwork Mon Aug 28 18:55:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 1826870 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.a=rsa-sha256 header.s=google header.b=FdiXYEtA; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RZKYB0xXxz1yZ9 for ; Tue, 29 Aug 2023 04:57:02 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qahP9-0000qu-7J; Mon, 28 Aug 2023 14:56:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qahP4-0000ot-7S for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:55:58 -0400 Received: from mail-oa1-x2b.google.com ([2001:4860:4864:20::2b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qahP1-0002Qz-Ac for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:55:57 -0400 Received: by mail-oa1-x2b.google.com with SMTP id 586e51a60fabf-1c8d895a602so2016419fac.2 for ; Mon, 28 Aug 2023 11:55:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1693248954; x=1693853754; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7K3agjAj/5Z9V2az8Pltf/TZABQcnS/GDuSWWPrto4M=; b=FdiXYEtAuIOeWw0FEEU4pTlgXX6bL0Bucwv+mbpNY3N7olA45QBU3ycKPPhDlTqKYL +LUCx6C2CRvewBDUnNccnmTafAnasLzTlyz6rGxRszJ52N/hZVE/GdPei/MhACYKPyst NqFbMUI6nJXf21SaZO7UtZVBkiIeSQDpTfuwenfPw442F4NsV8J3euXN6oUYkfIm9ykA JJOahdO0azAo75+T21ikOUxPBKx+/xwLfusSDYjyvG4U7iB38fwPBSn3wUbi//JB9tqk 2Zxytxxyw8bB2kyOklbhhsVU0zoheysg9hCGX7rnfjL3fwj21k3blKQungl6rA9ywk4q mWKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693248954; x=1693853754; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7K3agjAj/5Z9V2az8Pltf/TZABQcnS/GDuSWWPrto4M=; b=LVYMT0Me/Vc/0mDtZTs1Y1ojVZ9hu9Hmo81x/C34ngbnLi/MNG6/m2tZKFH06EG5e2 pInovv97vIr+v7sXKcxFd9K2OIMSpnqt6oiSvi3lnjVigzFgKwcbBFxjqsuFp9KmD5N/ BYzbyIi22gQeJPCkluJhI+ZGR7JGPmzdP84FP8pcLQx2sKot1UREieif7Bln5aoNaGnd vveZXjJAKtbt2yGcka1NB7wPXm3phEF+891mebsTSAAgEWXUTTCauncNumUlrnqPEu90 r7VVMY57hEAQwRn6HYXgDYSAExub1VCgYt/fdhJsfXDmO0DLsRcRHp2s4IsTqGDH+4KC 9WLg== X-Gm-Message-State: AOJu0YxCkqMsVQrvIrIZZVa0CQgm8sPrVU2WKK17jYFKG/OHC9B3i/lR NBJEoXK7/wdqgNt8xWmUXPFNeSPjf6nqiugNNqQ= X-Google-Smtp-Source: AGHT+IFNQcdw+N0i3wGp7ws/X/WbgJBPFV9FeL0ytDt+qIUzIsIXP2dXFP4M0QNr39+a79IdQPPBzg== X-Received: by 2002:a05:6870:4706:b0:1c8:bfd1:ecc0 with SMTP id b6-20020a056870470600b001c8bfd1ecc0mr12897140oaq.0.1693248954008; Mon, 28 Aug 2023 11:55:54 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id 4-20020a17090a190400b002680dfd368dsm7932835pjg.51.2023.08.28.11.55.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Aug 2023 11:55:53 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: alex.bennee@linaro.org Subject: [PATCH 02/10] accel/tcg: Split out io_prepare and io_failed Date: Mon, 28 Aug 2023 11:55:42 -0700 Message-Id: <20230828185550.573653-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230828185550.573653-1-richard.henderson@linaro.org> References: <20230828185550.573653-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2001:4860:4864:20::2b; envelope-from=richard.henderson@linaro.org; helo=mail-oa1-x2b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org These are common code from io_readx and io_writex. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- accel/tcg/cputlb.c | 77 +++++++++++++++++++++++++++------------------- 1 file changed, 45 insertions(+), 32 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index fcf6ccebff..17987f74e5 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1264,7 +1264,7 @@ void tlb_set_page_full(CPUState *cpu, int mmu_idx, * (non-page-aligned) vaddr of the eventual memory access to get * the MemoryRegion offset for the access. Note that the vaddr we * subtract here is that of the page base, and not the same as the - * vaddr we add back in io_readx()/io_writex()/get_page_addr_code(). + * vaddr we add back in io_prepare()/get_page_addr_code(). */ desc->fulltlb[index] = *full; full = &desc->fulltlb[index]; @@ -1364,37 +1364,60 @@ static inline void cpu_transaction_failed(CPUState *cpu, hwaddr physaddr, } } -static uint64_t io_readx(CPUArchState *env, CPUTLBEntryFull *full, - int mmu_idx, vaddr addr, uintptr_t retaddr, - MMUAccessType access_type, MemOp op) +static MemoryRegionSection * +io_prepare(hwaddr *out_offset, CPUArchState *env, hwaddr xlat, + MemTxAttrs attrs, vaddr addr, uintptr_t retaddr) { CPUState *cpu = env_cpu(env); - hwaddr mr_offset; MemoryRegionSection *section; - MemoryRegion *mr; - uint64_t val; - MemTxResult r; + hwaddr mr_offset; - section = iotlb_to_section(cpu, full->xlat_section, full->attrs); - mr = section->mr; - mr_offset = (full->xlat_section & TARGET_PAGE_MASK) + addr; + section = iotlb_to_section(cpu, xlat, attrs); + mr_offset = (xlat & TARGET_PAGE_MASK) + addr; cpu->mem_io_pc = retaddr; if (!cpu->can_do_io) { cpu_io_recompile(cpu, retaddr); } + *out_offset = mr_offset; + return section; +} + +static void io_failed(CPUArchState *env, CPUTLBEntryFull *full, vaddr addr, + unsigned size, MMUAccessType access_type, int mmu_idx, + MemTxResult response, uintptr_t retaddr, + MemoryRegionSection *section, hwaddr mr_offset) +{ + hwaddr physaddr = (mr_offset + + section->offset_within_address_space - + section->offset_within_region); + + cpu_transaction_failed(env_cpu(env), physaddr, addr, size, access_type, + mmu_idx, full->attrs, response, retaddr); +} + +static uint64_t io_readx(CPUArchState *env, CPUTLBEntryFull *full, + int mmu_idx, vaddr addr, uintptr_t retaddr, + MMUAccessType access_type, MemOp op) +{ + MemoryRegionSection *section; + hwaddr mr_offset; + MemoryRegion *mr; + MemTxResult r; + uint64_t val; + + section = io_prepare(&mr_offset, env, full->xlat_section, + full->attrs, addr, retaddr); + mr = section->mr; + { QEMU_IOTHREAD_LOCK_GUARD(); r = memory_region_dispatch_read(mr, mr_offset, &val, op, full->attrs); } if (r != MEMTX_OK) { - hwaddr physaddr = mr_offset + - section->offset_within_address_space - - section->offset_within_region; - - cpu_transaction_failed(cpu, physaddr, addr, memop_size(op), access_type, - mmu_idx, full->attrs, r, retaddr); + io_failed(env, full, addr, memop_size(op), access_type, mmu_idx, + r, retaddr, section, mr_offset); } return val; } @@ -1403,19 +1426,14 @@ static void io_writex(CPUArchState *env, CPUTLBEntryFull *full, int mmu_idx, uint64_t val, vaddr addr, uintptr_t retaddr, MemOp op) { - CPUState *cpu = env_cpu(env); - hwaddr mr_offset; MemoryRegionSection *section; + hwaddr mr_offset; MemoryRegion *mr; MemTxResult r; - section = iotlb_to_section(cpu, full->xlat_section, full->attrs); + section = io_prepare(&mr_offset, env, full->xlat_section, + full->attrs, addr, retaddr); mr = section->mr; - mr_offset = (full->xlat_section & TARGET_PAGE_MASK) + addr; - if (!cpu->can_do_io) { - cpu_io_recompile(cpu, retaddr); - } - cpu->mem_io_pc = retaddr; { QEMU_IOTHREAD_LOCK_GUARD(); @@ -1423,13 +1441,8 @@ static void io_writex(CPUArchState *env, CPUTLBEntryFull *full, } if (r != MEMTX_OK) { - hwaddr physaddr = mr_offset + - section->offset_within_address_space - - section->offset_within_region; - - cpu_transaction_failed(cpu, physaddr, addr, memop_size(op), - MMU_DATA_STORE, mmu_idx, full->attrs, r, - retaddr); + io_failed(env, full, addr, memop_size(op), MMU_DATA_STORE, mmu_idx, + r, retaddr, section, mr_offset); } } From patchwork Mon Aug 28 18:55:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 1826879 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.a=rsa-sha256 header.s=google header.b=Kd1cLMLD; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RZKZP4bXJz1yZ9 for ; Tue, 29 Aug 2023 04:58:05 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qahP6-0000pP-45; Mon, 28 Aug 2023 14:56:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qahP4-0000p1-R3 for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:55:58 -0400 Received: from mail-pg1-x52d.google.com ([2607:f8b0:4864:20::52d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qahP2-0002RV-Hk for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:55:58 -0400 Received: by mail-pg1-x52d.google.com with SMTP id 41be03b00d2f7-564b8e60ce9so1664984a12.2 for ; Mon, 28 Aug 2023 11:55:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1693248955; x=1693853755; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PXC5tTcUi0lmpfhMw90VKckGWB1iYRWY7/22QuukN5Y=; b=Kd1cLMLDbJlPQlmcLcuLYjtkG7ROMUJoSVcjc1Y2yY5ZivWdgP8FQHmF9bOonW8V/I J3mTuwSYQdYiU0uXQ9XowQI1ar6WrwLCtrAW/HJhlfEqfsCyB8QKztt7fnuoKYgDDs/O nq7jao+hCs/DKd4gHQosqLRrouZHv5QzCJNf3Rh90S8kAxXBLNg1gV0k9rGpYBzVBQWZ RQfvulioYJmPjzISsCbsKxhYgwofAgx5wfKRNxwEW3KYhqXB1iUhmoD+/FATUmHYvFYb MPFdODnk+sJO841ANvbNzh0EgbQACY1Pfi5c7BnCFT1SdTqWjzKXGF4Up9M8qQgkMxAZ 0MRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693248955; x=1693853755; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PXC5tTcUi0lmpfhMw90VKckGWB1iYRWY7/22QuukN5Y=; b=XV7TAwt8+0WYiha2uY8rSibtFhaQ3P2ALFGZs05gRkqOVyd8igikFvt9Uxj2j/RjM0 tscKccj+TWp8iWtHXWzlHxYssyxNGIwHp9O+sGi2liACoLixtro6ujJ4KKD+dxBjzu61 rgTp77EzZ9Mx1HNACDsJra37DzCNNEpkJawDxDC9OGGYh18v9dPZnR9QUoTCNzF/ofJj 0P7J8di2lLj8YKh4CbyaphwAosyQ8aWGCZx8WYAZRROoQORgt5NaoOG5DGHvy9uoX/tw VoV4j3iLy436jo28rX8yz0KPxtqChNnAwCQWYVX//OWFr9aajz0MqsXvcSFrABR8sF5a ZzJQ== X-Gm-Message-State: AOJu0Ywf7/Nml3O7ZqL4TvGrhPswdif5Y19EMHESyGBHY/pfkLOCkYMB rOh7VVBMs6ZFqcTOWN8nS4YfmzaSh56oV8sQU8o= X-Google-Smtp-Source: AGHT+IGgaSz+6lEpxa/DVXUokG50CdASoPWkhsEva/eVzM8g5bSfCuArgQo/dmi7sAPc+LTiAVBCyg== X-Received: by 2002:a17:90a:2f47:b0:268:fc26:73a9 with SMTP id s65-20020a17090a2f4700b00268fc2673a9mr19645961pjd.40.1693248955001; Mon, 28 Aug 2023 11:55:55 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id 4-20020a17090a190400b002680dfd368dsm7932835pjg.51.2023.08.28.11.55.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Aug 2023 11:55:54 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: alex.bennee@linaro.org Subject: [PATCH 03/10] accel/tcg: Use CPUTLBEntryFull.phys_addr in io_failed Date: Mon, 28 Aug 2023 11:55:43 -0700 Message-Id: <20230828185550.573653-4-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230828185550.573653-1-richard.henderson@linaro.org> References: <20230828185550.573653-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52d; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Since the introduction of CPUTLBEntryFull, we can recover the full cpu address space physical address without having to examine the MemoryRegionSection. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- accel/tcg/cputlb.c | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 17987f74e5..fd6c956214 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1385,13 +1385,9 @@ io_prepare(hwaddr *out_offset, CPUArchState *env, hwaddr xlat, static void io_failed(CPUArchState *env, CPUTLBEntryFull *full, vaddr addr, unsigned size, MMUAccessType access_type, int mmu_idx, - MemTxResult response, uintptr_t retaddr, - MemoryRegionSection *section, hwaddr mr_offset) + MemTxResult response, uintptr_t retaddr) { - hwaddr physaddr = (mr_offset + - section->offset_within_address_space - - section->offset_within_region); - + hwaddr physaddr = full->phys_addr | (addr & ~TARGET_PAGE_MASK); cpu_transaction_failed(env_cpu(env), physaddr, addr, size, access_type, mmu_idx, full->attrs, response, retaddr); } @@ -1417,7 +1413,7 @@ static uint64_t io_readx(CPUArchState *env, CPUTLBEntryFull *full, if (r != MEMTX_OK) { io_failed(env, full, addr, memop_size(op), access_type, mmu_idx, - r, retaddr, section, mr_offset); + r, retaddr); } return val; } @@ -1442,7 +1438,7 @@ static void io_writex(CPUArchState *env, CPUTLBEntryFull *full, if (r != MEMTX_OK) { io_failed(env, full, addr, memop_size(op), MMU_DATA_STORE, mmu_idx, - r, retaddr, section, mr_offset); + r, retaddr); } } From patchwork Mon Aug 28 18:55:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 1826876 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.a=rsa-sha256 header.s=google header.b=E4xlrPgE; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RZKYm2tNnz1yhW for ; Tue, 29 Aug 2023 04:57:32 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qahP6-0000pX-Q3; Mon, 28 Aug 2023 14:56:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qahP5-0000pN-UW for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:55:59 -0400 Received: from mail-pg1-x531.google.com ([2607:f8b0:4864:20::531]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qahP3-0002Rj-KD for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:55:59 -0400 Received: by mail-pg1-x531.google.com with SMTP id 41be03b00d2f7-56a55c0f8b1so1702578a12.0 for ; Mon, 28 Aug 2023 11:55:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1693248956; x=1693853756; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7fAz4IDlZXVLjSd/SH1PWPNAHXfaCucW3CyJSYpT2lM=; b=E4xlrPgE61OLpG+Lg2pYYPFU10HAxkOaVX5Qhs+/nyHFH20ZK74yUdx6Jv+GLDg9NI 2coQT4DL2VRySF6Og5j2of+lFXce+Y1DwP9299VXmTdfmVkGkdTb8dLiaAm+g286db3K H/etax29NYtIkQFLcxNEqISPLMFcyHdwWo24x7q7mn5Zsm93D+xX04pQ+ZBAAAARwWb3 LmhTLFJ8PnlTdIxTw4mx0SHxOfE9U3AYhdzkL11hROr6IfirNK/rUVZWwiiJ9hkVj8hO b7u004dRc2Pw1K3y/H14NFDsSrZHGpBd6mADd9A3DEHhFJvef0z46hhDJoqtksUxQb7o LfVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693248956; x=1693853756; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7fAz4IDlZXVLjSd/SH1PWPNAHXfaCucW3CyJSYpT2lM=; b=QfAO/P1M1ctzgJJLLBrPYsqASkZnmyDhBOEVhe2DBol/tJdjJM+KaMSg/K8HQ21E3X xKl4VXWAFujh239CiXnou1K4vaHOueWYmOV2I6D/FPR2Ts1+j2W9crMkIpwjyot1RLBu Bi3jz3LAnRUdDqSFbnVwQcpGFTuBWPCEauJSfFcEBqQEYqqUolUYkfkqHfVQGuvCjYbm mIZQ7Om0i0LSDw+9hOegL6aSArCoLvRHevEKIb85iKaZE6LxwmC2FgOw/UEtkkLX4tHb ji07glYO1XVY5LDqL2dZmc6oqjraVUNoDktShpkYbvFmtmO/WWlzIpv0tXokOrGXXRuM n1Rg== X-Gm-Message-State: AOJu0YxlKU4x68pOh8Xz10weYsvGBGZG/q7YXYMiIrodc/VTGYoIl7q0 CtOfbxc2AP34X6fz+2ZjhuRFcZljS8Q+U/mT+wg= X-Google-Smtp-Source: AGHT+IFCCgJ7k9yXS2Cyije0XaKUwDCwTOzbJ1U3n/3wWPJXpyQvtJqsb4xQ/3yv7hPkKv2HL/gJ0g== X-Received: by 2002:a17:90a:db08:b0:26b:3f10:820f with SMTP id g8-20020a17090adb0800b0026b3f10820fmr19825065pjv.36.1693248956126; Mon, 28 Aug 2023 11:55:56 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id 4-20020a17090a190400b002680dfd368dsm7932835pjg.51.2023.08.28.11.55.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Aug 2023 11:55:55 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: alex.bennee@linaro.org Subject: [PATCH 04/10] plugin: Simplify struct qemu_plugin_hwaddr Date: Mon, 28 Aug 2023 11:55:44 -0700 Message-Id: <20230828185550.573653-5-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230828185550.573653-1-richard.henderson@linaro.org> References: <20230828185550.573653-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::531; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x531.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Rather than saving MemoryRegionSection and offset, save phys_addr and MemoryRegion. This matches up much closer with the plugin api. Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- include/qemu/plugin-memory.h | 11 ++--------- accel/tcg/cputlb.c | 16 +++++++++------- plugins/api.c | 27 ++++++--------------------- 3 files changed, 17 insertions(+), 37 deletions(-) diff --git a/include/qemu/plugin-memory.h b/include/qemu/plugin-memory.h index 43165f2452..71c1123308 100644 --- a/include/qemu/plugin-memory.h +++ b/include/qemu/plugin-memory.h @@ -15,15 +15,8 @@ struct qemu_plugin_hwaddr { bool is_io; bool is_store; - union { - struct { - MemoryRegionSection *section; - hwaddr offset; - } io; - struct { - void *hostaddr; - } ram; - } v; + hwaddr phys_addr; + MemoryRegion *mr; }; /** diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index fd6c956214..b1dc213675 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1721,23 +1721,25 @@ bool tlb_plugin_lookup(CPUState *cpu, vaddr addr, int mmu_idx, uintptr_t index = tlb_index(env, mmu_idx, addr); MMUAccessType access_type = is_store ? MMU_DATA_STORE : MMU_DATA_LOAD; uint64_t tlb_addr = tlb_read_idx(tlbe, access_type); + CPUTLBEntryFull *full; if (unlikely(!tlb_hit(tlb_addr, addr))) { return false; } + full = &env_tlb(env)->d[mmu_idx].fulltlb[index]; + data->phys_addr = full->phys_addr | (addr & ~TARGET_PAGE_MASK); + /* We must have an iotlb entry for MMIO */ if (tlb_addr & TLB_MMIO) { - CPUTLBEntryFull *full = &env_tlb(env)->d[mmu_idx].fulltlb[index]; - hwaddr xlat = full->xlat_section; - + MemoryRegionSection *section = + iotlb_to_section(cpu, full->xlat_section & ~TARGET_PAGE_MASK, + full->attrs); data->is_io = true; - data->v.io.offset = (xlat & TARGET_PAGE_MASK) + addr; - data->v.io.section = - iotlb_to_section(cpu, xlat & ~TARGET_PAGE_MASK, full->attrs); + data->mr = section->mr; } else { data->is_io = false; - data->v.ram.hostaddr = (void *)((uintptr_t)addr + tlbe->addend); + data->mr = NULL; } return true; } diff --git a/plugins/api.c b/plugins/api.c index 2078b16edb..5521b0ad36 100644 --- a/plugins/api.c +++ b/plugins/api.c @@ -316,22 +316,7 @@ uint64_t qemu_plugin_hwaddr_phys_addr(const struct qemu_plugin_hwaddr *haddr) { #ifdef CONFIG_SOFTMMU if (haddr) { - if (!haddr->is_io) { - RAMBlock *block; - ram_addr_t offset; - void *hostaddr = haddr->v.ram.hostaddr; - - block = qemu_ram_block_from_host(hostaddr, false, &offset); - if (!block) { - error_report("Bad host ram pointer %p", haddr->v.ram.hostaddr); - abort(); - } - - return block->offset + offset + block->mr->addr; - } else { - MemoryRegionSection *mrs = haddr->v.io.section; - return mrs->offset_within_address_space + haddr->v.io.offset; - } + return haddr->phys_addr; } #endif return 0; @@ -341,13 +326,13 @@ const char *qemu_plugin_hwaddr_device_name(const struct qemu_plugin_hwaddr *h) { #ifdef CONFIG_SOFTMMU if (h && h->is_io) { - MemoryRegionSection *mrs = h->v.io.section; - if (!mrs->mr->name) { - unsigned long maddr = 0xffffffff & (uintptr_t) mrs->mr; - g_autofree char *temp = g_strdup_printf("anon%08lx", maddr); + MemoryRegion *mr = h->mr; + if (!mr->name) { + unsigned maddr = (uintptr_t)mr; + g_autofree char *temp = g_strdup_printf("anon%08x", maddr); return g_intern_string(temp); } else { - return g_intern_string(mrs->mr->name); + return g_intern_string(mr->name); } } else { return g_intern_static_string("RAM"); From patchwork Mon Aug 28 18:55:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 1826874 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.a=rsa-sha256 header.s=google header.b=lN7aAqol; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RZKYS0gM2z1yhW for ; Tue, 29 Aug 2023 04:57:16 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qahP8-0000qm-Sg; Mon, 28 Aug 2023 14:56:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qahP7-0000po-9D for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:56:01 -0400 Received: from mail-pj1-x102f.google.com ([2607:f8b0:4864:20::102f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qahP5-0002S2-3K for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:56:01 -0400 Received: by mail-pj1-x102f.google.com with SMTP id 98e67ed59e1d1-26b44247123so2254482a91.2 for ; Mon, 28 Aug 2023 11:55:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1693248957; x=1693853757; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zs4ouB+IX/j6ZyzVg7pnpseLfh71GLbrppugYTGLrDA=; b=lN7aAqolA/8tcvFHR0Zvln/tX1gsiAtzxgz8PfQaTFRmO4Q2G1WExradIaVLUIKdHL lnWGtKE0VB4kOMgeg5CMLaKdkUQ8q+sfyrG5hF6moKywqNAx4TaQQJTI15iSsVwFbG+7 rIduSO6Jx2kCErACxo/K4oiuS8QRlBEnPOUKneSYWL1hMbeEmgFcsmnnbaxVU680J17n kOQuW+aGs19UGzBY2gSMA4jr0Fw87cgiLsDFPlOTUYJPTWlnItLcmW+dBfsn0bYuC+cO etuExAJEsDvP9ox0qw2WQ0QkRAy0/HSKy1945M4j/9yyN6/SoO4p97VSsT0hgmLMZBXA g3jQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693248957; x=1693853757; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zs4ouB+IX/j6ZyzVg7pnpseLfh71GLbrppugYTGLrDA=; b=V+U5b9noFalV2T2A32Y82ZdKVqrkFMa0GccHckzQ3u8FBtx7/NECWrtHtUPoy7rerV 19PZ7/6AsPhXZUlakNEQ2vn/g862kFfYYGGTg9Fnpe08393okogMxpKNgpE6tVEl0Ztq fhgdKb1d3c9H0tuvEHgulbILk1PkjEJ2be8eIEc+3gGDYnubezoHVXIOZXBcZZB/BD14 nFU6xZ958zWHgAXaEOK70aKycVaqQ3e5kkOGGIfGMNDgRVotlUz6C5PMQR61h874Dw+f Ej7kbZa7S9IibVYTKYvxdY4JLbPIPK1hl4QotLX8jvJS65PQhlTq+4wl053P85Yh7yAG y46g== X-Gm-Message-State: AOJu0YzE1ZhzN9H+IRq35X0BO8nra1rDpX1ixLkmjZdLhxbbXFXJ3dUE trDiY1W/AgnUMU0hTfbEsePkKSZJhElt4BnxBC8= X-Google-Smtp-Source: AGHT+IFzDI2HViV8nPd3B9S2CPNog0xZI3BM1ZOSsFsSB/wuoj8HM+8LB3y/FihbQt+5NJgIwj9KeQ== X-Received: by 2002:a17:90b:4015:b0:262:e3aa:fd73 with SMTP id ie21-20020a17090b401500b00262e3aafd73mr25314055pjb.17.1693248957185; Mon, 28 Aug 2023 11:55:57 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id 4-20020a17090a190400b002680dfd368dsm7932835pjg.51.2023.08.28.11.55.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Aug 2023 11:55:56 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: alex.bennee@linaro.org Subject: [PATCH 05/10] accel/tcg: Merge cpu_transaction_failed into io_failed Date: Mon, 28 Aug 2023 11:55:45 -0700 Message-Id: <20230828185550.573653-6-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230828185550.573653-1-richard.henderson@linaro.org> References: <20230828185550.573653-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102f; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Push computation down into the if statements to the point the data is used. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- accel/tcg/cputlb.c | 33 +++++++++++++-------------------- 1 file changed, 13 insertions(+), 20 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index b1dc213675..d2e4c4459d 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1347,23 +1347,6 @@ static inline void cpu_unaligned_access(CPUState *cpu, vaddr addr, mmu_idx, retaddr); } -static inline void cpu_transaction_failed(CPUState *cpu, hwaddr physaddr, - vaddr addr, unsigned size, - MMUAccessType access_type, - int mmu_idx, MemTxAttrs attrs, - MemTxResult response, - uintptr_t retaddr) -{ - CPUClass *cc = CPU_GET_CLASS(cpu); - - if (!cpu->ignore_memory_transaction_failures && - cc->tcg_ops->do_transaction_failed) { - cc->tcg_ops->do_transaction_failed(cpu, physaddr, addr, size, - access_type, mmu_idx, attrs, - response, retaddr); - } -} - static MemoryRegionSection * io_prepare(hwaddr *out_offset, CPUArchState *env, hwaddr xlat, MemTxAttrs attrs, vaddr addr, uintptr_t retaddr) @@ -1387,9 +1370,19 @@ static void io_failed(CPUArchState *env, CPUTLBEntryFull *full, vaddr addr, unsigned size, MMUAccessType access_type, int mmu_idx, MemTxResult response, uintptr_t retaddr) { - hwaddr physaddr = full->phys_addr | (addr & ~TARGET_PAGE_MASK); - cpu_transaction_failed(env_cpu(env), physaddr, addr, size, access_type, - mmu_idx, full->attrs, response, retaddr); + CPUState *cpu = env_cpu(env); + + if (!cpu->ignore_memory_transaction_failures) { + CPUClass *cc = CPU_GET_CLASS(cpu); + + if (cc->tcg_ops->do_transaction_failed) { + hwaddr physaddr = full->phys_addr | (addr & ~TARGET_PAGE_MASK); + + cc->tcg_ops->do_transaction_failed(cpu, physaddr, addr, size, + access_type, mmu_idx, + full->attrs, response, retaddr); + } + } } static uint64_t io_readx(CPUArchState *env, CPUTLBEntryFull *full, From patchwork Mon Aug 28 18:55:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 1826875 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.a=rsa-sha256 header.s=google header.b=PVlt6LU7; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RZKYl4C07z1yZ9 for ; Tue, 29 Aug 2023 04:57:31 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qahP9-0000r1-Bn; Mon, 28 Aug 2023 14:56:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qahP7-0000pr-D3 for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:56:01 -0400 Received: from mail-pj1-x1032.google.com ([2607:f8b0:4864:20::1032]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qahP5-0002SC-9Q for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:56:01 -0400 Received: by mail-pj1-x1032.google.com with SMTP id 98e67ed59e1d1-268bc714ce0so3097399a91.0 for ; Mon, 28 Aug 2023 11:55:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1693248958; x=1693853758; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kes4dWAfnCF0sbsmuI+kEptn1hS/yp74fg4t9duR770=; b=PVlt6LU7ElyNwkbZTuIdBKWHN+yZ6AbiKL44WN+tpuTzvnCE7I7CoIGJ56vbugw01Q MX9ThoXSE2u0HkHbIJOGiTbabNx0NVFe3bv77/NJHQks6qAOdLc4vHJTHoicedeyyp7p +qeTA/UiRchYgzr5zFpuVfxIROdOHKXvW5nfbxJOMlXw2L/7v6j9yQhPlZQQmWulQJIA 3QQudAU2YgC/BL8+POFnEBIa3cTzFxvmUiUZBDsvDNUGzBQil9Tx0OakHg8mjvo51Q1i aH7hVo86Uid02hMEue3P3tJMZVwSBio9idOU+EqjN1vspAiyjeRPtuPIoCAb6UQWMwIz x+Ng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693248958; x=1693853758; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kes4dWAfnCF0sbsmuI+kEptn1hS/yp74fg4t9duR770=; b=d37QsIAX1/RDrrpw5OdYKqxB+n95e5vV3OTDL4c7HEm4zqYbOBGVCELSIvLqnf5p2i bXnum9WeTjf82MR98kQXLxb5nWHpmgmolnZgy389mj6P15mIST7xiUiNnmjuH0Tc13gK 2HrdmZ4lzbmBsa61uLrnoimJDdIbEAsjxZAbNPdM0O5x+8AaU0p4eXJp/jxahxQyXJ8V geQIO+ZcNEOsRdjD5QhsEXub6SUvTHutXvxFGGcriVU9t6YF8RUZ+xFmQBPYsB5goyrd x5BAvgvtohM5MC5iMjPDvMMnvaId+U7WoTQSaWKQae5qonLgcb8J+R8PiHnomVNdLYy7 Mmaw== X-Gm-Message-State: AOJu0YxUaHmkfbny2W6HqPFQgSK16aoZMkdVDpD0KoNyxB9sj/60WJaD odP3on/jOr4Nnt+AXtgdbmnD2cV35vk/vgg5P0I= X-Google-Smtp-Source: AGHT+IHQhAuWnC2xkH68grGmKg8HMs71amh8ABd5xowlhuTkGwZt/iiaYRjyBOKXIl7EAjAAfx8erA== X-Received: by 2002:a17:90a:d185:b0:268:808:8e82 with SMTP id fu5-20020a17090ad18500b0026808088e82mr683519pjb.1.1693248958105; Mon, 28 Aug 2023 11:55:58 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id 4-20020a17090a190400b002680dfd368dsm7932835pjg.51.2023.08.28.11.55.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Aug 2023 11:55:57 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: alex.bennee@linaro.org Subject: [PATCH 06/10] accel/tcg: Replace direct use of io_readx/io_writex in do_{ld, st}_1 Date: Mon, 28 Aug 2023 11:55:46 -0700 Message-Id: <20230828185550.573653-7-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230828185550.573653-1-richard.henderson@linaro.org> References: <20230828185550.573653-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1032; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1032.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Signed-off-by: Richard Henderson --- accel/tcg/cputlb.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index d2e4c4459d..6a7f6bf701 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -2339,7 +2339,8 @@ static uint8_t do_ld_1(CPUArchState *env, MMULookupPageData *p, int mmu_idx, MMUAccessType type, uintptr_t ra) { if (unlikely(p->flags & TLB_MMIO)) { - return io_readx(env, p->full, mmu_idx, p->addr, ra, type, MO_UB); + QEMU_IOTHREAD_LOCK_GUARD(); + return do_ld_mmio_beN(env, p->full, 0, p->addr, 1, mmu_idx, type, ra); } else { return *(uint8_t *)p->haddr; } @@ -2854,7 +2855,8 @@ static void do_st_1(CPUArchState *env, MMULookupPageData *p, uint8_t val, int mmu_idx, uintptr_t ra) { if (unlikely(p->flags & TLB_MMIO)) { - io_writex(env, p->full, mmu_idx, val, p->addr, ra, MO_UB); + QEMU_IOTHREAD_LOCK_GUARD(); + do_st_mmio_leN(env, p->full, val, p->addr, 1, mmu_idx, ra); } else if (unlikely(p->flags & TLB_DISCARD_WRITE)) { /* nothing */ } else { From patchwork Mon Aug 28 18:55:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 1826877 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.a=rsa-sha256 header.s=google header.b=ev9AcvLo; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RZKZ35lrRz1yZ9 for ; Tue, 29 Aug 2023 04:57:47 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qahPB-0000rp-FR; Mon, 28 Aug 2023 14:56:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qahP8-0000qn-Uo for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:56:02 -0400 Received: from mail-pj1-x1036.google.com ([2607:f8b0:4864:20::1036]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qahP6-0002SP-Hf for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:56:02 -0400 Received: by mail-pj1-x1036.google.com with SMTP id 98e67ed59e1d1-26d54d3d984so1646919a91.1 for ; Mon, 28 Aug 2023 11:56:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1693248959; x=1693853759; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8mz6E+/wbBE5eFwC0E1gYDTq5fpYP+E2DwEXnYpjQuQ=; b=ev9AcvLorCgsfvAPYLODTxwYRpY0ernGMzjQB++gjzdCOF7m/7rUDNg8MtMbyTUjpG sEQoIiYdyYa/VdZuCFiin5MtkDuIY7PIiDrz7ayWegFUQRwSLQOWwSZySn1ibVk9GGLN b/NgkvRG7k8kcx7gTBm8m447g9RLjaSxdj907kleSk4IdghmJK8x1e8CSe4vTEj+c7Yq 4XW2AsVLMtcY+0B3gETglvPay6SszOF158e89Q6vQSrT9nbuRQu/hCKcXdKS/hS5f+bi iBqmQKxSAjbQlcXyn3QCMJxpJ4mSnJzs9cecIJGKmc03QTwobRvbCUd8/tKVBXjGs7kS gegQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693248959; x=1693853759; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8mz6E+/wbBE5eFwC0E1gYDTq5fpYP+E2DwEXnYpjQuQ=; b=AJwr0nPKL2k6DTHFLB8CevzXLgpqtNv425UCqv5pPX15wXaCEYjY/bW1S0Id0arNKn Uq3vtTzb71CRCwTBjz6+zyzN4McbNq7/2wAV7j0gSteKEdTmfdhfmClzyKv/XLyfYUrn fe76HAFuoOMVtnODpuz07qyzrfAjP2Tv355GtLyTeDiC0pBcAIgPij5TuGcyzqHtpa+T u9ygfEyDMsiorvON8kpdhSpiG42FkTMNzIfPY2l4MsFs7on1xPY8IaK9jwYSRXmWwu6c 3mKFA+6+LhDeo93lj80xvn6pfY8RqbBbsjbdu9N6cqE3AJXEvc4Bx59b4EkYAmK23jCT Es7A== X-Gm-Message-State: AOJu0YyDfVdi/mmWmldmRqRHnkpkDBi+TX50jqyDHcwIViqLUC+cmqPN 1xwC1VzS/wARUAkzYbQcf95ap/Jo5RJZxhFlTWU= X-Google-Smtp-Source: AGHT+IEr8Qx5d/fi8EQl9DjRP972Nn9e/2ecbFR5JUM9YEXWwO2F6F3+lJ2O8cRcvUjdte6ve/WXsw== X-Received: by 2002:a17:90b:4a46:b0:26d:4421:854d with SMTP id lb6-20020a17090b4a4600b0026d4421854dmr20741968pjb.37.1693248959019; Mon, 28 Aug 2023 11:55:59 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id 4-20020a17090a190400b002680dfd368dsm7932835pjg.51.2023.08.28.11.55.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Aug 2023 11:55:58 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: alex.bennee@linaro.org Subject: [PATCH 07/10] accel/tcg: Merge io_readx into do_ld_mmio_beN Date: Mon, 28 Aug 2023 11:55:47 -0700 Message-Id: <20230828185550.573653-8-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230828185550.573653-1-richard.henderson@linaro.org> References: <20230828185550.573653-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1036; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1036.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Avoid multiple calls to io_prepare for unaligned acceses. One call to do_ld_mmio_beN will never cross pages. Signed-off-by: Richard Henderson --- accel/tcg/cputlb.c | 84 +++++++++++++++++----------------------------- 1 file changed, 30 insertions(+), 54 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 6a7f6bf701..f4772b5c5d 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1385,32 +1385,6 @@ static void io_failed(CPUArchState *env, CPUTLBEntryFull *full, vaddr addr, } } -static uint64_t io_readx(CPUArchState *env, CPUTLBEntryFull *full, - int mmu_idx, vaddr addr, uintptr_t retaddr, - MMUAccessType access_type, MemOp op) -{ - MemoryRegionSection *section; - hwaddr mr_offset; - MemoryRegion *mr; - MemTxResult r; - uint64_t val; - - section = io_prepare(&mr_offset, env, full->xlat_section, - full->attrs, addr, retaddr); - mr = section->mr; - - { - QEMU_IOTHREAD_LOCK_GUARD(); - r = memory_region_dispatch_read(mr, mr_offset, &val, op, full->attrs); - } - - if (r != MEMTX_OK) { - io_failed(env, full, addr, memop_size(op), access_type, mmu_idx, - r, retaddr); - } - return val; -} - static void io_writex(CPUArchState *env, CPUTLBEntryFull *full, int mmu_idx, uint64_t val, vaddr addr, uintptr_t retaddr, MemOp op) @@ -2059,40 +2033,42 @@ static uint64_t do_ld_mmio_beN(CPUArchState *env, CPUTLBEntryFull *full, uint64_t ret_be, vaddr addr, int size, int mmu_idx, MMUAccessType type, uintptr_t ra) { - uint64_t t; + MemoryRegionSection *section; + hwaddr mr_offset; + MemoryRegion *mr; + MemTxAttrs attrs; tcg_debug_assert(size > 0 && size <= 8); + + attrs = full->attrs; + section = io_prepare(&mr_offset, env, full->xlat_section, attrs, addr, ra); + mr = section->mr; + do { + MemOp this_mop; + unsigned this_size; + uint64_t val; + MemTxResult r; + /* Read aligned pieces up to 8 bytes. */ - switch ((size | (int)addr) & 7) { - case 1: - case 3: - case 5: - case 7: - t = io_readx(env, full, mmu_idx, addr, ra, type, MO_UB); - ret_be = (ret_be << 8) | t; - size -= 1; - addr += 1; - break; - case 2: - case 6: - t = io_readx(env, full, mmu_idx, addr, ra, type, MO_BEUW); - ret_be = (ret_be << 16) | t; - size -= 2; - addr += 2; - break; - case 4: - t = io_readx(env, full, mmu_idx, addr, ra, type, MO_BEUL); - ret_be = (ret_be << 32) | t; - size -= 4; - addr += 4; - break; - case 0: - return io_readx(env, full, mmu_idx, addr, ra, type, MO_BEUQ); - default: - qemu_build_not_reached(); + this_mop = ctz32(size | (int)addr | 8); + this_size = 1 << this_mop; + this_mop |= MO_BE; + + r = memory_region_dispatch_read(mr, mr_offset, &val, this_mop, attrs); + if (unlikely(r != MEMTX_OK)) { + io_failed(env, full, addr, this_size, type, mmu_idx, r, ra); } + if (this_size == 8) { + return val; + } + + ret_be = (ret_be << (this_size * 8)) | val; + addr += this_size; + mr_offset += this_size; + size -= this_size; } while (size); + return ret_be; } From patchwork Mon Aug 28 18:55:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 1826878 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.a=rsa-sha256 header.s=google header.b=gDR4y1GM; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RZKZD41SMz1yZ9 for ; Tue, 29 Aug 2023 04:57:56 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qahPB-0000rs-H0; Mon, 28 Aug 2023 14:56:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qahP9-0000rQ-L6 for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:56:03 -0400 Received: from mail-pj1-x102f.google.com ([2607:f8b0:4864:20::102f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qahP7-0002Sa-CZ for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:56:03 -0400 Received: by mail-pj1-x102f.google.com with SMTP id 98e67ed59e1d1-26ef24b8e5aso2282742a91.0 for ; Mon, 28 Aug 2023 11:56:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1693248960; x=1693853760; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Q52YryY77MZ2LmC1AwinyEGqkvB+2AbO6eV6neuXedY=; b=gDR4y1GM4p6ymFuZdvFKhpQUY8CeIu7itJ0Mo7YMk0FABEwHdlpawQbZVADkHdH/5T z6wQvAOJAkkwHewMYqcS57bXmoHKD3twileIqCmBpWxNxE05df77N07bvBps1heApkI0 WE3awanwadjzqIEKoT6wLEveu8/XtyXVq5eaxXNWmUslrB35xXvt+TCMsr9ap3Fvechm cPqZCEVvHKk3ybhsv33VCOHCBdrme0on7O8iJxpuq87fG3LINaHOF5Jv9gibolZKWvrV Z6ImsIHfwotvWbMEE8gFehq7iOeI2Isgptzy+GdvIpxfxnmv09Wq52c66NvUHC0YV49H Zfwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693248960; x=1693853760; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Q52YryY77MZ2LmC1AwinyEGqkvB+2AbO6eV6neuXedY=; b=Izv2/t9bfY9+5+w7imj/LA0YBPj9UpccrJ9/BkfHm1oJUkSEonYD4T5sUs1rYLj4RA Qr+wnq7f5jS+9YSNB/tsHDKFxq0ffpU5V9qCVhxg3dwy04xWXefXiq5YyaJR5k+iNKMq FXHYCaK5Ij7Cpd2Sw1gSfMgEmkjrw461kHG3sQWAHJ+TUbh0LeTv9YgI4HGcDlwMkmTs LZ2qIVfuPtIG2gryjQIVuGUvwE1l0UOs0/MOKWjryGplAchBhXyVLQdGHfJNvgWkpj6p /cR8yhO2pt3RvM8X98mGu81aTO1bGIEZrxRX40BkUV/JBPyRMMcSU6fOQp+VuDxxdwJ5 wWgw== X-Gm-Message-State: AOJu0YxWnIO393GSBk6FPO/6szUL+dvMP64EBbF1ounRvEuspw9kooYE wNcxwn+IljAC1MaqPnS1s1ssXNGM7FycCh9Zltg= X-Google-Smtp-Source: AGHT+IGxBMIFEKlp4ugpfLZz3ZnaamjU4yn718eki0bVDlrDF0tsv499IMFE2CpcRLswMUoCpUeslA== X-Received: by 2002:a17:90a:b395:b0:26f:6a0c:8210 with SMTP id e21-20020a17090ab39500b0026f6a0c8210mr21014529pjr.41.1693248959965; Mon, 28 Aug 2023 11:55:59 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id 4-20020a17090a190400b002680dfd368dsm7932835pjg.51.2023.08.28.11.55.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Aug 2023 11:55:59 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: alex.bennee@linaro.org Subject: [PATCH 08/10] accel/tcg: Merge io_writex into do_st_mmio_leN Date: Mon, 28 Aug 2023 11:55:48 -0700 Message-Id: <20230828185550.573653-9-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230828185550.573653-1-richard.henderson@linaro.org> References: <20230828185550.573653-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102f; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Avoid multiple calls to io_prepare for unaligned acceses. One call to do_st_mmio_leN will never cross pages. Signed-off-by: Richard Henderson --- accel/tcg/cputlb.c | 82 +++++++++++++++++----------------------------- 1 file changed, 30 insertions(+), 52 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index f4772b5c5d..cf80add4b3 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1385,30 +1385,6 @@ static void io_failed(CPUArchState *env, CPUTLBEntryFull *full, vaddr addr, } } -static void io_writex(CPUArchState *env, CPUTLBEntryFull *full, - int mmu_idx, uint64_t val, vaddr addr, - uintptr_t retaddr, MemOp op) -{ - MemoryRegionSection *section; - hwaddr mr_offset; - MemoryRegion *mr; - MemTxResult r; - - section = io_prepare(&mr_offset, env, full->xlat_section, - full->attrs, addr, retaddr); - mr = section->mr; - - { - QEMU_IOTHREAD_LOCK_GUARD(); - r = memory_region_dispatch_write(mr, mr_offset, val, op, full->attrs); - } - - if (r != MEMTX_OK) { - io_failed(env, full, addr, memop_size(op), MMU_DATA_STORE, mmu_idx, - r, retaddr); - } -} - /* Return true if ADDR is present in the victim tlb, and has been copied back to the main tlb. */ static bool victim_tlb_hit(CPUArchState *env, size_t mmu_idx, size_t index, @@ -2679,39 +2655,41 @@ static uint64_t do_st_mmio_leN(CPUArchState *env, CPUTLBEntryFull *full, uint64_t val_le, vaddr addr, int size, int mmu_idx, uintptr_t ra) { + MemoryRegionSection *section; + hwaddr mr_offset; + MemoryRegion *mr; + MemTxAttrs attrs; + tcg_debug_assert(size > 0 && size <= 8); + attrs = full->attrs; + section = io_prepare(&mr_offset, env, full->xlat_section, attrs, addr, ra); + mr = section->mr; + do { + MemOp this_mop; + unsigned this_size; + MemTxResult r; + /* Store aligned pieces up to 8 bytes. */ - switch ((size | (int)addr) & 7) { - case 1: - case 3: - case 5: - case 7: - io_writex(env, full, mmu_idx, val_le, addr, ra, MO_UB); - val_le >>= 8; - size -= 1; - addr += 1; - break; - case 2: - case 6: - io_writex(env, full, mmu_idx, val_le, addr, ra, MO_LEUW); - val_le >>= 16; - size -= 2; - addr += 2; - break; - case 4: - io_writex(env, full, mmu_idx, val_le, addr, ra, MO_LEUL); - val_le >>= 32; - size -= 4; - addr += 4; - break; - case 0: - io_writex(env, full, mmu_idx, val_le, addr, ra, MO_LEUQ); - return 0; - default: - qemu_build_not_reached(); + this_mop = ctz32(size | (int)addr | 8); + this_size = 1 << this_mop; + this_mop |= MO_LE; + + r = memory_region_dispatch_write(mr, mr_offset, val_le, + this_mop, attrs); + if (unlikely(r != MEMTX_OK)) { + io_failed(env, full, addr, this_size, MMU_DATA_STORE, + mmu_idx, r, ra); } + if (this_size == 8) { + return 0; + } + + val_le >>= this_size * 8; + addr += this_size; + mr_offset += this_size; + size -= this_size; } while (size); return val_le; From patchwork Mon Aug 28 18:55:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 1826872 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.a=rsa-sha256 header.s=google header.b=EKjbNHz4; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RZKYF3hJSz1yZ9 for ; Tue, 29 Aug 2023 04:57:05 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qahPC-0000sc-2c; Mon, 28 Aug 2023 14:56:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qahPA-0000rb-J9 for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:56:04 -0400 Received: from mail-pj1-x102f.google.com ([2607:f8b0:4864:20::102f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qahP8-0002Sn-1z for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:56:04 -0400 Received: by mail-pj1-x102f.google.com with SMTP id 98e67ed59e1d1-26d50a832a9so1857731a91.3 for ; Mon, 28 Aug 2023 11:56:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1693248961; x=1693853761; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cS4vlm8y6MqQ8ErQLX0t/96y+gcpjdiVv6UfrIIxZbo=; b=EKjbNHz4SXq67JnJFEotkal0Y0+t1ZmKBS6vctZQ/TqHPTRP/MLDQW+jW/xCgqM9zs 0yYOQjmO/j9nKf479bbUWaqfSVcQVL63KD8nd9dEBGaWPjnhc7arqpasVxmee87ypz4f mdnPnzSO+bX1JvPvPxJxItmzNu3v/gDeLJFsEfhzXbz2r73mNDYg4Kbaoij0Fd3htMI3 1xRpkzjknjPsWEUiDGODr2thB2Q27MMr7wyKma2W5cfRs0Lm8CKyWFpDVevmYFTZ3vS4 e0QTVvqMW4Z6Vw66C15olGS0lUQD94VXSVlJcCBs5PHKkTHj6R/p0+HxHw10NNBgEoi6 KK9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693248961; x=1693853761; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cS4vlm8y6MqQ8ErQLX0t/96y+gcpjdiVv6UfrIIxZbo=; b=Vn+RcMfyBDCvqwh4PVstCXGwUgbMEOfO5AUoS1JYVHaXIkcButLES+rLBvf+oL6dU/ e5qvPpFc6BY2GovoGH+wyBTvyD9khid5s8gNHvvVeaGpvYRxB5Gcns6OlaOMzbDgOlYq mJfgTVnI5fAcuRT6vOrI98r2U1L9xJzlI6Qm8xFlzAB+cvW2qBujBf5DYLAj5fAtEopO kz5KnRUvtLVvaTllDzHenrSXKS7M7WfuVQL4pGF+EHIvbarGwAxDpX9eVZ0LiXF4ZNNp KvyYD8gxgrwzuizU+Ul17WX6T/a8l6+6bakyBrFiMmVp3YN/Z6KJT4GZWr/0evT0k4VT 11wg== X-Gm-Message-State: AOJu0YzvmfHkZeuYvDKYfF7rAj42WwAxq1l0z2e/XNfhPwko4UcdZIEY mStrO1SR6+XQ2tw+DOCe9U7hI8VhKI0hRig95oo= X-Google-Smtp-Source: AGHT+IFE27Ke4hGLJvqIY4R9VBDmCNoSaH4hvyVlsoccKZNiwk+qgZuAySfFzKF5v2ds9CDwpgcUog== X-Received: by 2002:a17:90b:3110:b0:26b:36dc:2f08 with SMTP id gc16-20020a17090b311000b0026b36dc2f08mr20714299pjb.46.1693248960851; Mon, 28 Aug 2023 11:56:00 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id 4-20020a17090a190400b002680dfd368dsm7932835pjg.51.2023.08.28.11.56.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Aug 2023 11:56:00 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: alex.bennee@linaro.org Subject: [PATCH 09/10] accel/tcg: Introduce do_ld16_mmio_beN Date: Mon, 28 Aug 2023 11:55:49 -0700 Message-Id: <20230828185550.573653-10-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230828185550.573653-1-richard.henderson@linaro.org> References: <20230828185550.573653-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102f; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Split out int_ld_mmio_beN, to be used by both do_ld_mmio_beN and do_ld16_mmio_beN. Move the locks down into the two functions, since each one now covers all accesses to once page. Signed-off-by: Richard Henderson --- accel/tcg/cputlb.c | 91 ++++++++++++++++++++++++++++++---------------- 1 file changed, 59 insertions(+), 32 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index cf80add4b3..8a7e386415 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -2005,21 +2005,11 @@ static void *atomic_mmu_lookup(CPUArchState *env, vaddr addr, MemOpIdx oi, * Load @size bytes from @addr, which is memory-mapped i/o. * The bytes are concatenated in big-endian order with @ret_be. */ -static uint64_t do_ld_mmio_beN(CPUArchState *env, CPUTLBEntryFull *full, - uint64_t ret_be, vaddr addr, int size, - int mmu_idx, MMUAccessType type, uintptr_t ra) +static uint64_t int_ld_mmio_beN(CPUArchState *env, CPUTLBEntryFull *full, + uint64_t ret_be, vaddr addr, int size, + int mmu_idx, MMUAccessType type, uintptr_t ra, + MemoryRegion *mr, hwaddr mr_offset) { - MemoryRegionSection *section; - hwaddr mr_offset; - MemoryRegion *mr; - MemTxAttrs attrs; - - tcg_debug_assert(size > 0 && size <= 8); - - attrs = full->attrs; - section = io_prepare(&mr_offset, env, full->xlat_section, attrs, addr, ra); - mr = section->mr; - do { MemOp this_mop; unsigned this_size; @@ -2031,7 +2021,8 @@ static uint64_t do_ld_mmio_beN(CPUArchState *env, CPUTLBEntryFull *full, this_size = 1 << this_mop; this_mop |= MO_BE; - r = memory_region_dispatch_read(mr, mr_offset, &val, this_mop, attrs); + r = memory_region_dispatch_read(mr, mr_offset, &val, + this_mop, full->attrs); if (unlikely(r != MEMTX_OK)) { io_failed(env, full, addr, this_size, type, mmu_idx, r, ra); } @@ -2048,6 +2039,56 @@ static uint64_t do_ld_mmio_beN(CPUArchState *env, CPUTLBEntryFull *full, return ret_be; } +static uint64_t do_ld_mmio_beN(CPUArchState *env, CPUTLBEntryFull *full, + uint64_t ret_be, vaddr addr, int size, + int mmu_idx, MMUAccessType type, uintptr_t ra) +{ + MemoryRegionSection *section; + MemoryRegion *mr; + hwaddr mr_offset; + MemTxAttrs attrs; + uint64_t ret; + + tcg_debug_assert(size > 0 && size <= 8); + + attrs = full->attrs; + section = io_prepare(&mr_offset, env, full->xlat_section, attrs, addr, ra); + mr = section->mr; + + qemu_mutex_lock_iothread(); + ret = int_ld_mmio_beN(env, full, ret_be, addr, size, mmu_idx, + type, ra, mr, mr_offset); + qemu_mutex_unlock_iothread(); + + return ret; +} + +static Int128 do_ld16_mmio_beN(CPUArchState *env, CPUTLBEntryFull *full, + uint64_t ret_be, vaddr addr, int size, + int mmu_idx, uintptr_t ra) +{ + MemoryRegionSection *section; + MemoryRegion *mr; + hwaddr mr_offset; + MemTxAttrs attrs; + uint64_t a, b; + + tcg_debug_assert(size > 8 && size <= 16); + + attrs = full->attrs; + section = io_prepare(&mr_offset, env, full->xlat_section, attrs, addr, ra); + mr = section->mr; + + qemu_mutex_lock_iothread(); + a = int_ld_mmio_beN(env, full, ret_be, addr, size - 8, mmu_idx, + MMU_DATA_LOAD, ra, mr, mr_offset); + b = int_ld_mmio_beN(env, full, ret_be, addr + size - 8, 8, mmu_idx, + MMU_DATA_LOAD, ra, mr, mr_offset + size - 8); + qemu_mutex_unlock_iothread(); + + return int128_make128(b, a); +} + /** * do_ld_bytes_beN * @p: translation parameters @@ -2190,7 +2231,6 @@ static uint64_t do_ld_beN(CPUArchState *env, MMULookupPageData *p, unsigned tmp, half_size; if (unlikely(p->flags & TLB_MMIO)) { - QEMU_IOTHREAD_LOCK_GUARD(); return do_ld_mmio_beN(env, p->full, ret_be, p->addr, p->size, mmu_idx, type, ra); } @@ -2241,12 +2281,7 @@ static Int128 do_ld16_beN(CPUArchState *env, MMULookupPageData *p, MemOp atom; if (unlikely(p->flags & TLB_MMIO)) { - QEMU_IOTHREAD_LOCK_GUARD(); - a = do_ld_mmio_beN(env, p->full, a, p->addr, size - 8, - mmu_idx, MMU_DATA_LOAD, ra); - b = do_ld_mmio_beN(env, p->full, 0, p->addr + 8, 8, - mmu_idx, MMU_DATA_LOAD, ra); - return int128_make128(b, a); + return do_ld16_mmio_beN(env, p->full, a, p->addr, size, mmu_idx, ra); } /* @@ -2291,7 +2326,6 @@ static uint8_t do_ld_1(CPUArchState *env, MMULookupPageData *p, int mmu_idx, MMUAccessType type, uintptr_t ra) { if (unlikely(p->flags & TLB_MMIO)) { - QEMU_IOTHREAD_LOCK_GUARD(); return do_ld_mmio_beN(env, p->full, 0, p->addr, 1, mmu_idx, type, ra); } else { return *(uint8_t *)p->haddr; @@ -2304,7 +2338,6 @@ static uint16_t do_ld_2(CPUArchState *env, MMULookupPageData *p, int mmu_idx, uint16_t ret; if (unlikely(p->flags & TLB_MMIO)) { - QEMU_IOTHREAD_LOCK_GUARD(); ret = do_ld_mmio_beN(env, p->full, 0, p->addr, 2, mmu_idx, type, ra); if ((memop & MO_BSWAP) == MO_LE) { ret = bswap16(ret); @@ -2325,7 +2358,6 @@ static uint32_t do_ld_4(CPUArchState *env, MMULookupPageData *p, int mmu_idx, uint32_t ret; if (unlikely(p->flags & TLB_MMIO)) { - QEMU_IOTHREAD_LOCK_GUARD(); ret = do_ld_mmio_beN(env, p->full, 0, p->addr, 4, mmu_idx, type, ra); if ((memop & MO_BSWAP) == MO_LE) { ret = bswap32(ret); @@ -2346,7 +2378,6 @@ static uint64_t do_ld_8(CPUArchState *env, MMULookupPageData *p, int mmu_idx, uint64_t ret; if (unlikely(p->flags & TLB_MMIO)) { - QEMU_IOTHREAD_LOCK_GUARD(); ret = do_ld_mmio_beN(env, p->full, 0, p->addr, 8, mmu_idx, type, ra); if ((memop & MO_BSWAP) == MO_LE) { ret = bswap64(ret); @@ -2505,12 +2536,8 @@ static Int128 do_ld16_mmu(CPUArchState *env, vaddr addr, crosspage = mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD, &l); if (likely(!crosspage)) { if (unlikely(l.page[0].flags & TLB_MMIO)) { - QEMU_IOTHREAD_LOCK_GUARD(); - a = do_ld_mmio_beN(env, l.page[0].full, 0, addr, 8, - l.mmu_idx, MMU_DATA_LOAD, ra); - b = do_ld_mmio_beN(env, l.page[0].full, 0, addr + 8, 8, - l.mmu_idx, MMU_DATA_LOAD, ra); - ret = int128_make128(b, a); + ret = do_ld16_mmio_beN(env, l.page[0].full, 0, addr, 16, + l.mmu_idx, ra); if ((l.memop & MO_BSWAP) == MO_LE) { ret = bswap128(ret); } From patchwork Mon Aug 28 18:55:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 1826873 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.a=rsa-sha256 header.s=google header.b=lMgrsc0O; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4RZKYR1Pmyz1yZ9 for ; Tue, 29 Aug 2023 04:57:15 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qahPC-0000sl-Ug; Mon, 28 Aug 2023 14:56:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qahPB-0000sG-Nk for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:56:05 -0400 Received: from mail-pj1-x102b.google.com ([2607:f8b0:4864:20::102b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qahP9-0002Sv-8d for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:56:05 -0400 Received: by mail-pj1-x102b.google.com with SMTP id 98e67ed59e1d1-26f6b2c8e80so2265934a91.1 for ; Mon, 28 Aug 2023 11:56:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1693248962; x=1693853762; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OOzEj3zxNXy+t9npCtUx1T/2sNWvJU9XlZEuj/GyRA4=; b=lMgrsc0OYG2gn7Jt4IJ6jHnLSC4cB+MpYs4BDa3LAlXfDCT7ZiJ1q3UQ921seQJXgp dem+w35mkzct0tDALgYZGZE9uZRP8jqR7pWveiehiJx+Wv8/PYWYJvnNmmPQuiwK915+ 4u6Sr2615UDsua+RjMoOrzaoHXRwBeZFPPVQk6bMWnalq+MvfTBZ21KbKaPilF8SUi1Y akg+OR7iWQwxCEj/4Sufy/PvTZxMoeax25hpZIgvBAGK7EJIDgsMhASKX3imzwV97T+i 8MWi75xZ84CLFlpNy/fPX5MLjuRMHAX1gY1VcBrKvraBrQKEGa8rxGJ90fXirXXOJedF s8bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693248962; x=1693853762; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OOzEj3zxNXy+t9npCtUx1T/2sNWvJU9XlZEuj/GyRA4=; b=D7heM06TF5N6SNlhO5T0SZcHqTjagdAgZFQ7nUvJXOUE9QDCBcxl/MHbkJG91qbfVL 8AYb6IgTQUMxAA81uYWqHDnFzV4Fk9Rt5sydhW+f3v2+pDYxubNA63fUJn2FrVh3fQ/6 umP4i2Sqkmhtla5NfD4OmdlP7BU/2mYRJbFwc/wB7GdD2l6WsseyOAc+/N+s8QqnX8A/ uPe8Ndi+ibPArfgC5zehimvILyeFwUVkBRUhJmcFXrevLdWBfX4r172afCOXa3VpDMbl JQV/xCzXM9wk+bkLZtD2GiZz4TVhbZBdG93PqxfN8csghGTu1tWFBW1/FPuJ4N5t4xfe UNDw== X-Gm-Message-State: AOJu0YySZE0WUEx10nq9K2h8RqjKQ/+BcSiCtHVxYbrx1RDA1KD6aR/6 LMXlorxaG/ErhSRcQ7fsFtnP80SxkMbpHYQYcbs= X-Google-Smtp-Source: AGHT+IH69Uf+oLYNzqdE1XdZsf0SBZIkzRkOTO+bXqrgsBV3OF7BHW7jcED5FkcWRhzSiZYdv69RyQ== X-Received: by 2002:a17:90b:1a89:b0:268:ca76:64a with SMTP id ng9-20020a17090b1a8900b00268ca76064amr21315364pjb.49.1693248961827; Mon, 28 Aug 2023 11:56:01 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id 4-20020a17090a190400b002680dfd368dsm7932835pjg.51.2023.08.28.11.56.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Aug 2023 11:56:01 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: alex.bennee@linaro.org Subject: [PATCH 10/10] accel/tcg: Introduce do_st16_mmio_leN Date: Mon, 28 Aug 2023 11:55:50 -0700 Message-Id: <20230828185550.573653-11-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230828185550.573653-1-richard.henderson@linaro.org> References: <20230828185550.573653-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102b; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Split out int_st_mmio_leN, to be used by both do_st_mmio_leN and do_st16_mmio_leN. Move the locks down into the two functions, since each one now covers all accesses to once page. Signed-off-by: Richard Henderson --- accel/tcg/cputlb.c | 88 ++++++++++++++++++++++++++++++---------------- 1 file changed, 58 insertions(+), 30 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 8a7e386415..ff253e78d2 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -2678,21 +2678,11 @@ Int128 cpu_ld16_mmu(CPUArchState *env, abi_ptr addr, * The bytes to store are extracted in little-endian order from @val_le; * return the bytes of @val_le beyond @p->size that have not been stored. */ -static uint64_t do_st_mmio_leN(CPUArchState *env, CPUTLBEntryFull *full, - uint64_t val_le, vaddr addr, int size, - int mmu_idx, uintptr_t ra) +static uint64_t int_st_mmio_leN(CPUArchState *env, CPUTLBEntryFull *full, + uint64_t val_le, vaddr addr, int size, + int mmu_idx, uintptr_t ra, + MemoryRegion *mr, hwaddr mr_offset) { - MemoryRegionSection *section; - hwaddr mr_offset; - MemoryRegion *mr; - MemTxAttrs attrs; - - tcg_debug_assert(size > 0 && size <= 8); - - attrs = full->attrs; - section = io_prepare(&mr_offset, env, full->xlat_section, attrs, addr, ra); - mr = section->mr; - do { MemOp this_mop; unsigned this_size; @@ -2704,7 +2694,7 @@ static uint64_t do_st_mmio_leN(CPUArchState *env, CPUTLBEntryFull *full, this_mop |= MO_LE; r = memory_region_dispatch_write(mr, mr_offset, val_le, - this_mop, attrs); + this_mop, full->attrs); if (unlikely(r != MEMTX_OK)) { io_failed(env, full, addr, this_size, MMU_DATA_STORE, mmu_idx, r, ra); @@ -2722,6 +2712,56 @@ static uint64_t do_st_mmio_leN(CPUArchState *env, CPUTLBEntryFull *full, return val_le; } +static uint64_t do_st_mmio_leN(CPUArchState *env, CPUTLBEntryFull *full, + uint64_t val_le, vaddr addr, int size, + int mmu_idx, uintptr_t ra) +{ + MemoryRegionSection *section; + hwaddr mr_offset; + MemoryRegion *mr; + MemTxAttrs attrs; + uint64_t ret; + + tcg_debug_assert(size > 0 && size <= 8); + + attrs = full->attrs; + section = io_prepare(&mr_offset, env, full->xlat_section, attrs, addr, ra); + mr = section->mr; + + qemu_mutex_lock_iothread(); + ret = int_st_mmio_leN(env, full, val_le, addr, size, mmu_idx, + ra, mr, mr_offset); + qemu_mutex_unlock_iothread(); + + return ret; +} + +static uint64_t do_st16_mmio_leN(CPUArchState *env, CPUTLBEntryFull *full, + Int128 val_le, vaddr addr, int size, + int mmu_idx, uintptr_t ra) +{ + MemoryRegionSection *section; + MemoryRegion *mr; + hwaddr mr_offset; + MemTxAttrs attrs; + uint64_t ret; + + tcg_debug_assert(size > 8 && size <= 16); + + attrs = full->attrs; + section = io_prepare(&mr_offset, env, full->xlat_section, attrs, addr, ra); + mr = section->mr; + + qemu_mutex_lock_iothread(); + int_st_mmio_leN(env, full, int128_getlo(val_le), addr, 8, + mmu_idx, ra, mr, mr_offset); + ret = int_st_mmio_leN(env, full, int128_gethi(val_le), addr + 8, + size - 8, mmu_idx, ra, mr, mr_offset + 8); + qemu_mutex_unlock_iothread(); + + return ret; +} + /* * Wrapper for the above. */ @@ -2733,7 +2773,6 @@ static uint64_t do_st_leN(CPUArchState *env, MMULookupPageData *p, unsigned tmp, half_size; if (unlikely(p->flags & TLB_MMIO)) { - QEMU_IOTHREAD_LOCK_GUARD(); return do_st_mmio_leN(env, p->full, val_le, p->addr, p->size, mmu_idx, ra); } else if (unlikely(p->flags & TLB_DISCARD_WRITE)) { @@ -2788,11 +2827,8 @@ static uint64_t do_st16_leN(CPUArchState *env, MMULookupPageData *p, MemOp atom; if (unlikely(p->flags & TLB_MMIO)) { - QEMU_IOTHREAD_LOCK_GUARD(); - do_st_mmio_leN(env, p->full, int128_getlo(val_le), - p->addr, 8, mmu_idx, ra); - return do_st_mmio_leN(env, p->full, int128_gethi(val_le), - p->addr + 8, size - 8, mmu_idx, ra); + return do_st16_mmio_leN(env, p->full, val_le, p->addr, + size, mmu_idx, ra); } else if (unlikely(p->flags & TLB_DISCARD_WRITE)) { return int128_gethi(val_le) >> ((size - 8) * 8); } @@ -2836,7 +2872,6 @@ static void do_st_1(CPUArchState *env, MMULookupPageData *p, uint8_t val, int mmu_idx, uintptr_t ra) { if (unlikely(p->flags & TLB_MMIO)) { - QEMU_IOTHREAD_LOCK_GUARD(); do_st_mmio_leN(env, p->full, val, p->addr, 1, mmu_idx, ra); } else if (unlikely(p->flags & TLB_DISCARD_WRITE)) { /* nothing */ @@ -2852,7 +2887,6 @@ static void do_st_2(CPUArchState *env, MMULookupPageData *p, uint16_t val, if ((memop & MO_BSWAP) != MO_LE) { val = bswap16(val); } - QEMU_IOTHREAD_LOCK_GUARD(); do_st_mmio_leN(env, p->full, val, p->addr, 2, mmu_idx, ra); } else if (unlikely(p->flags & TLB_DISCARD_WRITE)) { /* nothing */ @@ -2872,7 +2906,6 @@ static void do_st_4(CPUArchState *env, MMULookupPageData *p, uint32_t val, if ((memop & MO_BSWAP) != MO_LE) { val = bswap32(val); } - QEMU_IOTHREAD_LOCK_GUARD(); do_st_mmio_leN(env, p->full, val, p->addr, 4, mmu_idx, ra); } else if (unlikely(p->flags & TLB_DISCARD_WRITE)) { /* nothing */ @@ -2892,7 +2925,6 @@ static void do_st_8(CPUArchState *env, MMULookupPageData *p, uint64_t val, if ((memop & MO_BSWAP) != MO_LE) { val = bswap64(val); } - QEMU_IOTHREAD_LOCK_GUARD(); do_st_mmio_leN(env, p->full, val, p->addr, 8, mmu_idx, ra); } else if (unlikely(p->flags & TLB_DISCARD_WRITE)) { /* nothing */ @@ -3020,11 +3052,7 @@ static void do_st16_mmu(CPUArchState *env, vaddr addr, Int128 val, if ((l.memop & MO_BSWAP) != MO_LE) { val = bswap128(val); } - a = int128_getlo(val); - b = int128_gethi(val); - QEMU_IOTHREAD_LOCK_GUARD(); - do_st_mmio_leN(env, l.page[0].full, a, addr, 8, l.mmu_idx, ra); - do_st_mmio_leN(env, l.page[0].full, b, addr + 8, 8, l.mmu_idx, ra); + do_st16_mmio_leN(env, l.page[0].full, val, addr, 16, l.mmu_idx, ra); } else if (unlikely(l.page[0].flags & TLB_DISCARD_WRITE)) { /* nothing */ } else {