[v1,0/7] target/riscv: Support SXL32 on RV64 CPU

From: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com>

We have implemented UXL32 on QEMU already. It enables us to run RV32
applications on RV64 Linux on QEMU. Similarly, RISCV specification
doesn't limit the SXLEN to be the same with MXLEN. In this patch set,
we will go forward to implement SXL32 on RV64 CPU.

SXL is one filed in mstatus. It defines the SXLEN. After CPU reset,
SXL will be same with MXL in misa.

According to the RISC-V privilege specification, Smode CSR with SXLEN
length will usually be read and written in an SXLEN view, no matter
current XLEN is 32bit or 64bit. Thus if MXLEN=64 and SXLEN=32, M mode
register will read or write them in the same way under RV32 layout.

In this patch set, we carefully process the Smode CSR write and read
for SXL32. This includes explicitly CSR write/read, and implicitly
CSR read such as satp read in PTW(page table walk).

Another thing we do in this patch set is that we introduce a property
of CPU, named sxl32. It provides us a way to override the default sxl64
for RV64 cpu after cpu reset. The reason is OpenSBI currently doesn't
dynamically probe or configure SXLEN. When we want to run RV64 OpenSBI
in M mode and RV32 Linux in S mode, OpenSBI will not prepare the SXL32
environment for RV32 Linux. sxl32 property will also forbidden writing SXL
field in mstatus, which means QEMU can ensure an SXL32 environment for Linux.

We've successfully booted 64-bit OpenSBI and 32-bit Linux system on
RV64 CPU, with a minor fix of Linux.

Tested by follow steps:

 - Prepare Linux rv32 Image && rv64 Image
  Please apply the patch of linux before test. You can get the patch in
https://github.com/romanheros/linux/commit/8444fb3c913d64c20b1e5334a29a66061eb3f5fe.

	$ cd linux
	$ make ARCH=riscv CROSS_COMPILE=riscv32-unknown-linux-gnu- O=../linux-rv32 defconfig
	$ make ARCH=riscv CROSS_COMPILE=riscv32-unknown-linux-gnu- O=../build-rv32 32-bit.config
	$ make ARCH=riscv CROSS_COMPILE=riscv32-unknown-linux-gnu- O=../linux-rv32/ -j $(nproc)
	$ make ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- O=../linux-rv64 defconfig
	$ make ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- O=../linux-rv64/ -j $(nproc)
	(Get Image in ../linux-rv32/arch/riscv/boot)

 - Prepare QEMU
 	$ git clone https://github.com/romanheros/qemu.git -b riscv-upstream-sxl32-v1
	$ cd qemu
	$ ./configure --target-list="riscv64-softmmu" --enable-slirp
	$ cd build; ninja

- Run 32-bit Linux boot on 64-bit RISC-V with Opensbi 64 :
	$ ./build/qemu-system-riscv64 -cpu rv64,sxl32=on -M virt -nographic \
		-bios opensbi/build/platform/generic/firmware/fw_dynamic.bin \
		-kernel linux-rv32/arch/riscv/boot/Image \
		-append "root=/dev/vda ro console=ttyS0" \
		-drive file=qemu_riscv32_virt_defconfig/images/rootfs.ext2,format=raw,id=hd0 \
		-netdev user,id=net0 -device virtio-net-device,netdev=net0
```
OpenSBI v1.4-112-g7b4bc9d
Build time: 2024-06-17 06:57:42 +0000
Build compiler: gcc version 13.2.0 (gc891d8dc23e-dirty)
   ____                    _____ ____ _____
  / __ \                  / ____|  _ \_   _|
 | |  | |_ __   ___ _ __ | (___ | |_) || |
 | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
 | |__| | |_) |  __/ | | |____) | |_) || |_
  \____/| .__/ \___|_| |_|_____/|____/_____|
        | |
        |_|

Platform Name             : riscv-virtio,qemu
Platform Features         : medeleg
Platform HART Count       : 1
Platform IPI Device       : aclint-mswi
Platform Timer Device     : aclint-mtimer @ 10000000Hz
Platform Console Device   : uart8250
Platform HSM Device       : ---
Platform PMU Device       : ---
Platform Reboot Device    : syscon-reboot
Platform Shutdown Device  : syscon-poweroff
Platform Suspend Device   : ---
Platform CPPC Device      : ---
Firmware Base             : 0x80000000
Firmware Size             : 583 KB
Firmware RW Offset        : 0x80000
Firmware RW Size          : 71 KB
Firmware Heap Offset      : 0x89000
Firmware Heap Size        : 35 KB (total), 2 KB (reserved), 11 KB (used), 21 KB (free)
Firmware Scratch Size     : 4096 B (total), 400 B (used), 3696 B (free)
Runtime SBI Version       : 2.0

Domain0 Name              : root
Domain0 Boot HART         : 0
Domain0 HARTs             : 0*
Domain0 Region00          : 0x0000000000100000-0x0000000000100fff M: (I,R,W) S/U: (R,W)
Domain0 Region01          : 0x0000000010000000-0x0000000010000fff M: (I,R,W) S/U: (R,W)
Domain0 Region02          : 0x0000000002000000-0x000000000200ffff M: (I,R,W) S/U: ()
Domain0 Region03          : 0x0000000080080000-0x000000008009ffff M: (R,W) S/U: ()
Domain0 Region04          : 0x0000000080000000-0x000000008007ffff M: (R,X) S/U: ()
Domain0 Region05          : 0x000000000c400000-0x000000000c5fffff M: (I,R,W) S/U: (R,W)
Domain0 Region06          : 0x000000000c000000-0x000000000c3fffff M: (I,R,W) S/U: (R,W)
Domain0 Region07          : 0x0000000000000000-0xffffffffffffffff M: () S/U: (R,W,X)
Domain0 Next Address      : 0x0000000080200000
Domain0 Next Arg1         : 0x0000000087e00000
Domain0 Next Mode         : S-mode
Domain0 SysReset          : yes
Domain0 SysSuspend        : yes

Boot HART ID              : 0
Boot HART Domain          : root
Boot HART Priv Version    : v1.12
Boot HART Base ISA        : rv64imafdch
Boot HART ISA Extensions  : sstc,zicntr,zihpm,zicboz,zicbom,sdtrig
Boot HART PMP Count       : 16
Boot HART PMP Granularity : 2 bits
Boot HART PMP Address Bits: 54
Boot HART MHPM Info       : 16 (0x0007fff8)
Boot HART Debug Triggers  : 2 triggers
Boot HART MIDELEG         : 0x0000000000001666
Boot HART MEDELEG         : 0x0000000000f0b509
[    0.000000] Linux version 6.10.0-rc3-00100-gec0abe37042f (developer@95e6e39b54aa) (riscv32-unknown-linux-gnu-gcc (gc891d8dc23e-dirty) 13.2.0, GNU ld (GNU Binutils) 2.42) #1 SMP Sat Jun 15 13:01:52 UTC 2024
[    0.000000] random: crng init done
[    0.000000] Machine model: riscv-virtio,qemu
[    0.000000] SBI specification v2.0 detected
[    0.000000] SBI implementation ID=0x1 Version=0x10004
[    0.000000] SBI TIME extension detected
[    0.000000] SBI IPI extension detected
[    0.000000] SBI RFENCE extension detected
[    0.000000] SBI SRST extension detected
[    0.000000] SBI DBCN extension detected
[    0.000000] efi: UEFI not found.
[    0.000000] OF: reserved mem: 0x0000000080000000..0x000000008007ffff (512 KiB) nomap non-reusable mmode_resv1@80000000
[    0.000000] OF: reserved mem: 0x0000000080080000..0x000000008009ffff (128 KiB) nomap non-reusable mmode_resv0@80080000
[    0.000000] Zone ranges:
[    0.000000]   DMA32    [mem 0x0000000080000000-0x0000000087ffffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000080000000-0x000000008009ffff]
[    0.000000]   node   0: [mem 0x00000000800a0000-0x0000000087ffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x0000000087ffffff]
[    0.000000] SBI HSM extension detected
[    0.000000] riscv: base ISA extensions acdfhim
[    0.000000] riscv: ELF capabilities acdfim
......
Starting network: udhcpc: started, v1.36.1
udhcpc: broadcasting discover
udhcpc: broadcasting select for 10.0.2.15, server 10.0.2.2
udhcpc: lease of 10.0.2.15 obtained from 10.0.2.2, lease time 86400
deleting routers
adding dns 10.0.2.3
OK

Welcome to Buildroot
buildroot login: root
# cat /proc/cpuinfo
processor       : 0
hart            : 0
isa             : rv64imafdch_zicbom_zicboz_zicntr_zicsr_zifencei_zihintntl_zihintpause_zihpm_zfa_zba_zbb_zbc_zbs_sstc
mmu             : sv57
mvendorid       : 0x0
marchid         : 0x0
mimpid          : 0x0
hart isa        : rv64imafdch_zicbom_zicboz_zicntr_zicsr_zifencei_zihintntl_zihintpause_zihpm_zfa_zba_zbb_zbc_zbs_sstc
```

This doesn't mean we have addressed all the challenges for this problem. The to do list is here:
1. OpenSBI need accept a parameter from firmware the next boot level is SXL32 instead of only S mode.
2. OpenSBI should dynamically configure the SXL32 environment for next boot.
3. OpenSBI should ensure SBI calls for RV32 are right implemented even M mode is RV64.
4. OpenSBI should override details of machine configuration for RV32 Linux, such as rv64i ISA description in DTB.

Having all of them been addressed, we can allow the SXL write in mstatus. But this todo list depends on
a runnable environment, which is provided by this patch set.

This patch set is based on the patch set[1] under review. You can also fetch the branch from my repo[2].

[1]: https://mail.gnu.org/archive/html/qemu-riscv/2024-09/msg00501.html
[2]: https://github.com/romanheros/qemu/tree/riscv-upstream-sxl32-v1

TANG Tiancheng (7):
  target/riscv: Fix sstatus read and write
  target/riscv: Fix satp read and write implicitly or explicitly.
  target/riscv: Read pte and satp based on SXL in PTW
  hw/riscv: Align kernel to 4MB when sxl32 is on.
  target/riscv: Enable 32-bit only registers for RV64 with sxl32
  target/riscv: Reset SXL and UXL according to sxl32
  target/riscv: Expose sxl32 configuration in RISC-V CPU

 hw/riscv/boot.c           |  3 ++-
 target/riscv/cpu.c        | 14 ++++++++++++--
 target/riscv/cpu_cfg.h    |  4 ++++
 target/riscv/cpu_helper.c |  4 ++--
 target/riscv/csr.c        | 38 +++++++++++++++++++++++++-------------
 5 files changed, 45 insertions(+), 18 deletions(-)