[v3] tests/avocado: add test to exercize processor address space memory bound checks

QEMU has validations to make sure that a VM is not started with more memory
(static and hotpluggable memory) than what the guest processor can address
directly with its addressing bits. This change adds a test to make sure QEMU
fails to start with a specific error message when an attempt is made to
start a VM with more memory than what the processor can directly address.
The test also checks for passing cases when the address space of the processor
is capable of addressing all memory. Boundary cases are tested.

CC: imammedo@redhat.com
CC: David Hildenbrand <david@redhat.com>
Signed-off-by: Ani Sinha <anisinha@redhat.com>
---
 tests/avocado/mem-addr-space-check.py | 331 ++++++++++++++++++++++++++
 1 file changed, 331 insertions(+)
 create mode 100644 tests/avocado/mem-addr-space-check.py

Changelog:
v3: added pae tests as well.
v2: added 64-bit tests. Added cxl tests.

Sample run:
$ ./pyvenv/bin/avocado run tests/avocado/mem-addr-space-check.py --tap -
1..14
ok 1 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_pse36
ok 2 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_pae
ok 3 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_pentium_pse36
ok 4 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_pentium_pae
ok 5 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_pentium2
ok 6 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_nonpse36
ok 7 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_tcg_q35_70_amd
ok 8 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_tcg_q35_71_amd
ok 9 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_tcg_q35_70_amd
ok 10 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_tcg_q35_71_amd
ok 11 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_tcg_q35_71_intel
ok 12 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_tcg_q35_71_amd_41bits
ok 13 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_tcg_q35_intel_cxl
ok 14 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_tcg_q35_intel_cxl

> On 27-Oct-2023, at 4:12 PM, Ani Sinha <anisinha@redhat.com> wrote:
> 
> QEMU has validations to make sure that a VM is not started with more memory
> (static and hotpluggable memory) than what the guest processor can address
> directly with its addressing bits. This change adds a test to make sure QEMU
> fails to start with a specific error message when an attempt is made to
> start a VM with more memory than what the processor can directly address.
> The test also checks for passing cases when the address space of the processor
> is capable of addressing all memory. Boundary cases are tested.
> 
> CC: imammedo@redhat.com
> CC: David Hildenbrand <david@redhat.com>
> Signed-off-by: Ani Sinha <anisinha@redhat.com>

Gentle ping on this. David does these tests look good and cover all cases more or less?

> ---
> tests/avocado/mem-addr-space-check.py | 331 ++++++++++++++++++++++++++
> 1 file changed, 331 insertions(+)
> create mode 100644 tests/avocado/mem-addr-space-check.py
> 
> Changelog:
> v3: added pae tests as well.
> v2: added 64-bit tests. Added cxl tests.
> 
> Sample run:
> $ ./pyvenv/bin/avocado run tests/avocado/mem-addr-space-check.py --tap -
> 1..14
> ok 1 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_pse36
> ok 2 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_pae
> ok 3 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_pentium_pse36
> ok 4 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_pentium_pae
> ok 5 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_pentium2
> ok 6 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_nonpse36
> ok 7 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_tcg_q35_70_amd
> ok 8 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_tcg_q35_71_amd
> ok 9 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_tcg_q35_70_amd
> ok 10 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_tcg_q35_71_amd
> ok 11 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_tcg_q35_71_intel
> ok 12 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_tcg_q35_71_amd_41bits
> ok 13 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_tcg_q35_intel_cxl
> ok 14 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_tcg_q35_intel_cxl
> 
> diff --git a/tests/avocado/mem-addr-space-check.py b/tests/avocado/mem-addr-space-check.py
> new file mode 100644
> index 0000000000..6ded11d658
> --- /dev/null
> +++ b/tests/avocado/mem-addr-space-check.py
> @@ -0,0 +1,331 @@
> +# Check for crash when using memory beyond the available guest processor
> +# address space.
> +#
> +# Copyright (c) 2023 Red Hat, Inc.
> +#
> +# Author:
> +#  Ani Sinha <anisinha@redhat.com>
> +#
> +# This program is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License as published by
> +# the Free Software Foundation; either version 2 of the License, or
> +# (at your option) any later version.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program.  If not, see <http://www.gnu.org/licenses/>.
> +
> +from avocado_qemu import QemuSystemTest
> +import signal
> +
> +class MemAddrCheck(QemuSystemTest):
> +    # first, lets test some 32-bit processors.
> +    # for all 32-bit cases, pci64_hole_size is 0.
> +    def test_phybits_low_pse36(self):
> +        """
> +        :avocado: tags=machine:q35
> +        :avocado: tags=arch:x86_64
> +
> +        With pse36 feature ON, a processor has 36 bits of addressing. So it can
> +        access up to a maximum of 64GiB of memory. Memory hotplug region begins
> +        at 4 GiB boundary when "above_4g_mem_size" is 0 (this would be true when
> +        we have 0.5 GiB of VM memory, see pc_q35_init()). This means total
> +        hotpluggable memory size is 60 GiB. Per slot, we reserve 1 GiB of memory
> +        for dimm alignment for all newer machines (see enforce_aligned_dimm
> +        property for pc machines and pc_get_device_memory_range()). That leaves
> +        total hotpluggable actual memory size of 59 GiB. If the VM is started
> +        with 0.5 GiB of memory, maxmem should be set to a maximum value of
> +        59.5 GiB to ensure that the processor can address all memory directly.
> +        Note that 64-bit pci hole size is 0 in this case. If maxmem is set to
> +        59.6G, QEMU should fail to start with a message "phy-bits are too low".
> +        If maxmem is set to 59.5G with all other QEMU parameters identical, QEMU
> +        should start fine.
> +        """
> +        self.vm.add_args('-S', '-machine', 'q35', '-m',
> +                         '512,slots=1,maxmem=59.6G',
> +                         '-cpu', 'pentium,pse36=on', '-display', 'none',
> +                         '-object', 'memory-backend-ram,id=mem1,size=1G',
> +                         '-device', 'virtio-mem-pci,id=vm0,memdev=mem1')
> +        self.vm.set_qmp_monitor(enabled=False)
> +        self.vm.launch()
> +        self.vm.wait()
> +        self.assertEquals(self.vm.exitcode(), 1, "QEMU exit code should be 1")
> +        self.assertRegex(self.vm.get_log(), r'phys-bits too low')
> +
> +    def test_phybits_low_pae(self):
> +        """
> +        :avocado: tags=machine:q35
> +        :avocado: tags=arch:x86_64
> +
> +        With pae feature ON, a processor has 36 bits of addressing. So it can
> +        access up to a maximum of 64GiB of memory. Rest is the same as the case
> +        with pse36 above.
> +        """
> +        self.vm.add_args('-S', '-machine', 'q35', '-m',
> +                         '512,slots=1,maxmem=59.6G',
> +                         '-cpu', 'pentium,pae=on', '-display', 'none',
> +                         '-object', 'memory-backend-ram,id=mem1,size=1G',
> +                         '-device', 'virtio-mem-pci,id=vm0,memdev=mem1')
> +        self.vm.set_qmp_monitor(enabled=False)
> +        self.vm.launch()
> +        self.vm.wait()
> +        self.assertEquals(self.vm.exitcode(), 1, "QEMU exit code should be 1")
> +        self.assertRegex(self.vm.get_log(), r'phys-bits too low')
> +
> +    def test_phybits_ok_pentium_pse36(self):
> +        """
> +        :avocado: tags=machine:q35
> +        :avocado: tags=arch:x86_64
> +
> +        Setting maxmem to 59.5G and making sure that QEMU can start with the
> +        same options as the failing case above with pse36 cpu feature.
> +        """
> +        self.vm.add_args('-machine', 'q35', '-m',
> +                         '512,slots=1,maxmem=59.5G',
> +                         '-cpu', 'pentium,pse36=on', '-display', 'none',
> +                         '-object', 'memory-backend-ram,id=mem1,size=1G',
> +                         '-device', 'virtio-mem-pci,id=vm0,memdev=mem1')
> +        self.vm.set_qmp_monitor(enabled=False)
> +        self.vm.launch()
> +        self.vm.shutdown()
> +        self.assertEquals(self.vm.exitcode(), -signal.SIGTERM,
> +                          "QEMU did not terminate gracefully upon SIGTERM")
> +
> +    def test_phybits_ok_pentium_pae(self):
> +        """
> +        :avocado: tags=machine:q35
> +        :avocado: tags=arch:x86_64
> +
> +        Test is same as above but now with pae CPUID turned on. Setting
> +        maxmem to 59.5G and making sure that QEMU can start fine with the
> +        same options as the case above.
> +        """
> +        self.vm.add_args('-machine', 'q35', '-m',
> +                         '512,slots=1,maxmem=59.5G',
> +                         '-cpu', 'pentium,pae=on', '-display', 'none',
> +                         '-object', 'memory-backend-ram,id=mem1,size=1G',
> +                         '-device', 'virtio-mem-pci,id=vm0,memdev=mem1')
> +        self.vm.set_qmp_monitor(enabled=False)
> +        self.vm.launch()
> +        self.vm.shutdown()
> +        self.assertEquals(self.vm.exitcode(), -signal.SIGTERM,
> +                          "QEMU did not terminate gracefully upon SIGTERM")
> +
> +    def test_phybits_ok_pentium2(self):
> +        """
> +        :avocado: tags=machine:q35
> +        :avocado: tags=arch:x86_64
> +
> +        Pentium2 has 36 bits of addressing, so its same as pentium
> +        with pse36 ON.
> +        """
> +        self.vm.add_args('-machine', 'q35', '-m',
> +                         '512,slots=1,maxmem=59.5G',
> +                         '-cpu', 'pentium2', '-display', 'none',
> +                         '-object', 'memory-backend-ram,id=mem1,size=1G',
> +                         '-device', 'virtio-mem-pci,id=vm0,memdev=mem1')
> +        self.vm.set_qmp_monitor(enabled=False)
> +        self.vm.launch()
> +        self.vm.shutdown()
> +        self.assertEquals(self.vm.exitcode(), -signal.SIGTERM,
> +                          "QEMU did not terminate gracefully upon SIGTERM")
> +
> +    def test_phybits_low_nonpse36(self):
> +        """
> +        :avocado: tags=machine:q35
> +        :avocado: tags=arch:x86_64
> +
> +        Pentium processor has 32 bits of addressing without pse36 or pae
> +        so it can access up to 4 GiB of memory. Setting maxmem to 4GiB
> +        should make QEMU fail to start with "phys-bits too low" message.
> +        """
> +        self.vm.add_args('-S', '-machine', 'q35', '-m',
> +                         '512,slots=1,maxmem=4G',
> +                         '-cpu', 'pentium', '-display', 'none',
> +                         '-object', 'memory-backend-ram,id=mem1,size=1G',
> +                         '-device', 'virtio-mem-pci,id=vm0,memdev=mem1')
> +        self.vm.set_qmp_monitor(enabled=False)
> +        self.vm.launch()
> +        self.vm.wait()
> +        self.assertEquals(self.vm.exitcode(), 1, "QEMU exit code should be 1")
> +        self.assertRegex(self.vm.get_log(), r'phys-bits too low')
> +
> +    # now lets test some 64-bit CPU cases.
> +    def test_phybits_low_tcg_q35_70_amd(self):
> +        """
> +        :avocado: tags=machine:q35
> +        :avocado: tags=arch:x86_64
> +
> +        For q35 7.1 machines and above, "above_4G" memory starts at 1 TiB
> +        boundary for AMD cpus (default). Lets test without that case.
> +        For q35-7.0 machines, "above 4G" memory starts are 4G.
> +        pci64_hole size is 32 GiB. Since TCG_PHYS_ADDR_BITS is defined to
> +        be 40, TCG emulated CPUs have maximum of 1 TiB (1024 GiB) of
> +        directly addressible memory.
> +        Hence, maxmem value at most can be
> +        1024 GiB - 4 GiB - 1 GiB per slot for alignment - 32 GiB + 0.5 GiB
> +        which is equal to 987.5 GiB. Setting the value to 988 GiB should
> +        make QEMU fail with the error message.
> +        """
> +        self.vm.add_args('-S', '-machine', 'pc-q35-7.0', '-m',
> +                         '512,slots=1,maxmem=988G',
> +                         '-display', 'none',
> +                         '-object', 'memory-backend-ram,id=mem1,size=1G',
> +                         '-device', 'virtio-mem-pci,id=vm0,memdev=mem1')
> +        self.vm.set_qmp_monitor(enabled=False)
> +        self.vm.launch()
> +        self.vm.wait()
> +        self.assertEquals(self.vm.exitcode(), 1, "QEMU exit code should be 1")
> +        self.assertRegex(self.vm.get_log(), r'phys-bits too low')
> +
> +    def test_phybits_low_tcg_q35_71_amd(self):
> +        """
> +        :avocado: tags=machine:q35
> +        :avocado: tags=arch:x86_64
> +
> +        AMD_HT_START is defined to be at 1012 GiB. So for q35 machines
> +        version > 7.0 and AMD cpus, instead of 1024 GiB limit for 40 bit
> +        processor address space, it has to be 1012 GiB , that is 12 GiB
> +        less than the case above.
> +        Make sure QEMU fails when maxmem size is 976 GiB (12 GiB less
> +        than 988 GiB).
> +        """
> +        self.vm.add_args('-S', '-machine', 'pc-q35-7.1', '-m',
> +                         '512,slots=1,maxmem=976G',
> +                         '-display', 'none',
> +                         '-object', 'memory-backend-ram,id=mem1,size=1G',
> +                         '-device', 'virtio-mem-pci,id=vm0,memdev=mem1')
> +        self.vm.set_qmp_monitor(enabled=False)
> +        self.vm.launch()
> +        self.vm.wait()
> +        self.assertEquals(self.vm.exitcode(), 1, "QEMU exit code should be 1")
> +        self.assertRegex(self.vm.get_log(), r'phys-bits too low')
> +
> +    def test_phybits_ok_tcg_q35_70_amd(self):
> +        """
> +        :avocado: tags=machine:q35
> +        :avocado: tags=arch:x86_64
> +
> +        Same as q35-7.0 AMD case except that here we check that QEMU can
> +        successfully start when maxmem is < 988G.
> +        """
> +        self.vm.add_args('-S', '-machine', 'pc-q35-7.0', '-m',
> +                         '512,slots=1,maxmem=987.5G',
> +                         '-display', 'none',
> +                         '-object', 'memory-backend-ram,id=mem1,size=1G',
> +                         '-device', 'virtio-mem-pci,id=vm0,memdev=mem1')
> +        self.vm.set_qmp_monitor(enabled=False)
> +        self.vm.launch()
> +        self.vm.shutdown()
> +        self.assertEquals(self.vm.exitcode(), -signal.SIGTERM,
> +                          "QEMU did not terminate gracefully upon SIGTERM")
> +
> +    def test_phybits_ok_tcg_q35_71_amd(self):
> +        """
> +        :avocado: tags=machine:q35
> +        :avocado: tags=arch:x86_64
> +
> +        Same as q35-7.1 AMD case except that here we check that QEMU can
> +        successfully start when maxmem is < 976G.
> +        """
> +        self.vm.add_args('-S', '-machine', 'pc-q35-7.1', '-m',
> +                         '512,slots=1,maxmem=975.5G',
> +                         '-display', 'none',
> +                         '-object', 'memory-backend-ram,id=mem1,size=1G',
> +                         '-device', 'virtio-mem-pci,id=vm0,memdev=mem1')
> +        self.vm.set_qmp_monitor(enabled=False)
> +        self.vm.launch()
> +        self.vm.shutdown()
> +        self.assertEquals(self.vm.exitcode(), -signal.SIGTERM,
> +                          "QEMU did not terminate gracefully upon SIGTERM")
> +
> +    def test_phybits_ok_tcg_q35_71_intel(self):
> +        """
> +        :avocado: tags=machine:q35
> +        :avocado: tags=arch:x86_64
> +
> +        Same parameters as test_phybits_low_tcg_q35_71_amd() but use
> +        Intel cpu instead. QEMU should start fine in this case as
> +        "above_4G" memory starts at 4G.
> +        """
> +        self.vm.add_args('-S', '-cpu', 'Skylake-Server',
> +                         '-machine', 'pc-q35-7.1', '-m',
> +                         '512,slots=1,maxmem=976G',
> +                         '-display', 'none',
> +                         '-object', 'memory-backend-ram,id=mem1,size=1G',
> +                         '-device', 'virtio-mem-pci,id=vm0,memdev=mem1')
> +        self.vm.set_qmp_monitor(enabled=False)
> +        self.vm.launch()
> +        self.vm.shutdown()
> +        self.assertEquals(self.vm.exitcode(), -signal.SIGTERM,
> +                          "QEMU did not terminate gracefully upon SIGTERM")
> +
> +    def test_phybits_low_tcg_q35_71_amd_41bits(self):
> +        """
> +        :avocado: tags=machine:q35
> +        :avocado: tags=arch:x86_64
> +
> +        AMD processor with 41 bits. Max cpu hw address = 2 TiB.
> +        "above_4G" memory starts at 1 TiB for q35-7.1 machines. So with
> +        pci_64_hole size at 32 GiB, maxmem should be 991.5 GiB with 1 GiB per
> +        slot for alignment and 0.5 GiB as static memory for the VM
> +        (1024 - 32 - 1 + 0.5). With 992 GiB, QEMU should fail to start.
> +        """
> +        self.vm.add_args('-S', '-cpu', 'EPYC-v4,phys-bits=41',
> +                         '-machine', 'pc-q35-7.1', '-m',
> +                         '512,slots=1,maxmem=992G',
> +                         '-display', 'none',
> +                         '-object', 'memory-backend-ram,id=mem1,size=1G',
> +                         '-device', 'virtio-mem-pci,id=vm0,memdev=mem1')
> +        self.vm.set_qmp_monitor(enabled=False)
> +        self.vm.launch()
> +        self.vm.wait()
> +        self.assertEquals(self.vm.exitcode(), 1, "QEMU exit code should be 1")
> +        self.assertRegex(self.vm.get_log(), r'phys-bits too low')
> +
> +    def test_phybits_low_tcg_q35_intel_cxl(self):
> +        """
> +        :avocado: tags=machine:q35
> +        :avocado: tags=arch:x86_64
> +
> +        cxl memory window starts after memory device range. Here, we use 1 GiB
> +        of cxl window memory. 4G_mem end aligns at 4G. pci64_hole is 32 GiB and
> +        starts after the cxl memory window.
> +        So maxmem here should be at most 986 GiB considering all memory boundary
> +        alignment constraints with 40 bits (1 TiB) of processor physical bits.
> +        """
> +        self.vm.add_args('-S', '-cpu', 'Skylake-Server,phys-bits=40',
> +                         '-machine', 'q35,cxl=on', '-m',
> +                         '512,slots=1,maxmem=987G',
> +                         '-display', 'none',
> +                         '-device', 'pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1',
> +                         '-M', 'cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=1G')
> +        self.vm.set_qmp_monitor(enabled=False)
> +        self.vm.launch()
> +        self.vm.wait()
> +        self.assertEquals(self.vm.exitcode(), 1, "QEMU exit code should be 1")
> +        self.assertRegex(self.vm.get_log(), r'phys-bits too low')
> +
> +    def test_phybits_ok_tcg_q35_intel_cxl(self):
> +        """
> +        :avocado: tags=machine:q35
> +        :avocado: tags=arch:x86_64
> +
> +        Same as above but here we do not reserve any cxl memory window. Hence,
> +        with the exact same parameters as above, QEMU should start fine even
> +        with cxl enabled.
> +        """
> +        self.vm.add_args('-S', '-cpu', 'Skylake-Server,phys-bits=40',
> +                         '-machine', 'q35,cxl=on', '-m',
> +                         '512,slots=1,maxmem=987G',
> +                         '-display', 'none',
> +                         '-device', 'pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1')
> +        self.vm.set_qmp_monitor(enabled=False)
> +        self.vm.launch()
> +        self.vm.shutdown()
> +        self.assertEquals(self.vm.exitcode(), -signal.SIGTERM,
> +                          "QEMU did not terminate gracefully upon SIGTERM")
> -- 
> 2.42.0
>

On 01.11.23 02:29, Ani Sinha wrote:
> 
> 
>> On 27-Oct-2023, at 4:12 PM, Ani Sinha <anisinha@redhat.com> wrote:
>>
>> QEMU has validations to make sure that a VM is not started with more memory
>> (static and hotpluggable memory) than what the guest processor can address
>> directly with its addressing bits. This change adds a test to make sure QEMU
>> fails to start with a specific error message when an attempt is made to
>> start a VM with more memory than what the processor can directly address.
>> The test also checks for passing cases when the address space of the processor
>> is capable of addressing all memory. Boundary cases are tested.
>>
>> CC: imammedo@redhat.com
>> CC: David Hildenbrand <david@redhat.com>
>> Signed-off-by: Ani Sinha <anisinha@redhat.com>
> 
> Gentle ping on this. David does these tests look good and cover all cases more or less?

Hi,

sorry, for some reason the patches never made it to my inbox.

> 
>> ---
>> tests/avocado/mem-addr-space-check.py | 331 ++++++++++++++++++++++++++
>> 1 file changed, 331 insertions(+)
>> create mode 100644 tests/avocado/mem-addr-space-check.py
>>
>> Changelog:
>> v3: added pae tests as well.
>> v2: added 64-bit tests. Added cxl tests.
>>
>> Sample run:
>> $ ./pyvenv/bin/avocado run tests/avocado/mem-addr-space-check.py --tap -
>> 1..14
>> ok 1 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_pse36
>> ok 2 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_pae
>> ok 3 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_pentium_pse36
>> ok 4 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_pentium_pae
>> ok 5 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_pentium2
>> ok 6 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_nonpse36
>> ok 7 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_tcg_q35_70_amd
>> ok 8 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_tcg_q35_71_amd
>> ok 9 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_tcg_q35_70_amd
>> ok 10 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_tcg_q35_71_amd
>> ok 11 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_tcg_q35_71_intel
>> ok 12 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_tcg_q35_71_amd_41bits
>> ok 13 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_tcg_q35_intel_cxl
>> ok 14 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_tcg_q35_intel_cxl
>>
>> diff --git a/tests/avocado/mem-addr-space-check.py b/tests/avocado/mem-addr-space-check.py
>> new file mode 100644
>> index 0000000000..6ded11d658
>> --- /dev/null
>> +++ b/tests/avocado/mem-addr-space-check.py
>> @@ -0,0 +1,331 @@
>> +# Check for crash when using memory beyond the available guest processor
>> +# address space.
>> +#
>> +# Copyright (c) 2023 Red Hat, Inc.
>> +#
>> +# Author:
>> +#  Ani Sinha <anisinha@redhat.com>
>> +#
>> +# This program is free software; you can redistribute it and/or modify
>> +# it under the terms of the GNU General Public License as published by
>> +# the Free Software Foundation; either version 2 of the License, or
>> +# (at your option) any later version.
>> +#
>> +# This program is distributed in the hope that it will be useful,
>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> +# GNU General Public License for more details.
>> +#
>> +# You should have received a copy of the GNU General Public License
>> +# along with this program.  If not, see <http://www.gnu.org/licenses/>.
>> +
>> +from avocado_qemu import QemuSystemTest
>> +import signal
>> +
>> +class MemAddrCheck(QemuSystemTest):
>> +    # first, lets test some 32-bit processors.
>> +    # for all 32-bit cases, pci64_hole_size is 0.
>> +    def test_phybits_low_pse36(self):
>> +        """
>> +        :avocado: tags=machine:q35
>> +        :avocado: tags=arch:x86_64
>> +
>> +        With pse36 feature ON, a processor has 36 bits of addressing. So it can
>> +        access up to a maximum of 64GiB of memory. Memory hotplug region begins
>> +        at 4 GiB boundary when "above_4g_mem_size" is 0 (this would be true when
>> +        we have 0.5 GiB of VM memory, see pc_q35_init()). This means total
>> +        hotpluggable memory size is 60 GiB. Per slot, we reserve 1 GiB of memory
>> +        for dimm alignment for all newer machines (see enforce_aligned_dimm
>> +        property for pc machines and pc_get_device_memory_range()). That leaves
>> +        total hotpluggable actual memory size of 59 GiB. If the VM is started
>> +        with 0.5 GiB of memory, maxmem should be set to a maximum value of
>> +        59.5 GiB to ensure that the processor can address all memory directly.
>> +        Note that 64-bit pci hole size is 0 in this case. If maxmem is set to
>> +        59.6G, QEMU should fail to start with a message "phy-bits are too low".
>> +        If maxmem is set to 59.5G with all other QEMU parameters identical, QEMU
>> +        should start fine.
>> +        """
>> +        self.vm.add_args('-S', '-machine', 'q35', '-m',
>> +                         '512,slots=1,maxmem=59.6G',
>> +                         '-cpu', 'pentium,pse36=on', '-display', 'none',
>> +                         '-object', 'memory-backend-ram,id=mem1,size=1G',
>> +                         '-device', 'virtio-mem-pci,id=vm0,memdev=mem1')

Just a note that for virtio-mem, you don't even have to reserve an (ACPI NVDIMM/DIMM) slot.
So you might get away just setting "slots=0" and using maxmem=60.6G,
making it possible to simplify the comment(s) a bit (and all the other test cases).

Otherwise, maybe clearer use a DIMM instead of a virtio-mem device.

>> +        self.vm.set_qmp_monitor(enabled=False)
>> +        self.vm.launch()
>> +        self.vm.wait()
>> +        self.assertEquals(self.vm.exitcode(), 1, "QEMU exit code should be 1")
>> +        self.assertRegex(self.vm.get_log(), r'phys-bits too low')
>> +
>> +    def test_phybits_low_pae(self):
>> +        """
>> +        :avocado: tags=machine:q35
>> +        :avocado: tags=arch:x86_64
>> +
>> +        With pae feature ON, a processor has 36 bits of addressing. So it can
>> +        access up to a maximum of 64GiB of memory. Rest is the same as the case
>> +        with pse36 above.
>> +        """
>> +        self.vm.add_args('-S', '-machine', 'q35', '-m',
>> +                         '512,slots=1,maxmem=59.6G',
>> +                         '-cpu', 'pentium,pae=on', '-display', 'none',
>> +                         '-object', 'memory-backend-ram,id=mem1,size=1G',
>> +                         '-device', 'virtio-mem-pci,id=vm0,memdev=mem1')

Same considerations.

>> +        self.vm.set_qmp_monitor(enabled=False)
>> +        self.vm.launch()
>> +        self.vm.wait()
>> +        self.assertEquals(self.vm.exitcode(), 1, "QEMU exit code should be 1")
>> +        self.assertRegex(self.vm.get_log(), r'phys-bits too low')
>> +
>> +    def test_phybits_ok_pentium_pse36(self):
>> +        """
>> +        :avocado: tags=machine:q35
>> +        :avocado: tags=arch:x86_64
>> +
>> +        Setting maxmem to 59.5G and making sure that QEMU can start with the
>> +        same options as the failing case above with pse36 cpu feature.
>> +        """
>> +        self.vm.add_args('-machine', 'q35', '-m',
>> +                         '512,slots=1,maxmem=59.5G',
>> +                         '-cpu', 'pentium,pse36=on', '-display', 'none',
>> +                         '-object', 'memory-backend-ram,id=mem1,size=1G',
>> +                         '-device', 'virtio-mem-pci,id=vm0,memdev=mem1')
>> +        self.vm.set_qmp_monitor(enabled=False)
>> +        self.vm.launch()
>> +        self.vm.shutdown()
>> +        self.assertEquals(self.vm.exitcode(), -signal.SIGTERM,
>> +                          "QEMU did not terminate gracefully upon SIGTERM")
>> +
>> +    def test_phybits_ok_pentium_pae(self):
>> +        """
>> +        :avocado: tags=machine:q35
>> +        :avocado: tags=arch:x86_64
>> +
>> +        Test is same as above but now with pae CPUID turned on. Setting

s/CPUID/cpu feture/ ?

>> +        maxmem to 59.5G and making sure that QEMU can start fine with the
>> +        same options as the case above.
>> +        """
>> +        self.vm.add_args('-machine', 'q35', '-m',
>> +                         '512,slots=1,maxmem=59.5G',
>> +                         '-cpu', 'pentium,pae=on', '-display', 'none',
>> +                         '-object', 'memory-backend-ram,id=mem1,size=1G',
>> +                         '-device', 'virtio-mem-pci,id=vm0,memdev=mem1')
>> +        self.vm.set_qmp_monitor(enabled=False)
>> +        self.vm.launch()
>> +        self.vm.shutdown()
>> +        self.assertEquals(self.vm.exitcode(), -signal.SIGTERM,
>> +                          "QEMU did not terminate gracefully upon SIGTERM")
>> +
>> +    def test_phybits_ok_pentium2(self):
>> +        """
>> +        :avocado: tags=machine:q35
>> +        :avocado: tags=arch:x86_64
>> +
>> +        Pentium2 has 36 bits of addressing, so its same as pentium
>> +        with pse36 ON.
>> +        """
>> +        self.vm.add_args('-machine', 'q35', '-m',
>> +                         '512,slots=1,maxmem=59.5G',
>> +                         '-cpu', 'pentium2', '-display', 'none',
>> +                         '-object', 'memory-backend-ram,id=mem1,size=1G',
>> +                         '-device', 'virtio-mem-pci,id=vm0,memdev=mem1')
>> +        self.vm.set_qmp_monitor(enabled=False)
>> +        self.vm.launch()
>> +        self.vm.shutdown()
>> +        self.assertEquals(self.vm.exitcode(), -signal.SIGTERM,
>> +                          "QEMU did not terminate gracefully upon SIGTERM")
>> +
>> +    def test_phybits_low_nonpse36(self):
>> +        """
>> +        :avocado: tags=machine:q35
>> +        :avocado: tags=arch:x86_64
>> +
>> +        Pentium processor has 32 bits of addressing without pse36 or pae
>> +        so it can access up to 4 GiB of memory. Setting maxmem to 4GiB

"access physical addresses up to 4 GiB"

>> +        should make QEMU fail to start with "phys-bits too low" message.

"because the region for memory hotplug is always placed above 4 GiB due to the
PCI hole and simplicity."

>> +        """
>> +        self.vm.add_args('-S', '-machine', 'q35', '-m',
>> +                         '512,slots=1,maxmem=4G',
>> +                         '-cpu', 'pentium', '-display', 'none',
>> +                         '-object', 'memory-backend-ram,id=mem1,size=1G',
>> +                         '-device', 'virtio-mem-pci,id=vm0,memdev=mem1')
>> +        self.vm.set_qmp_monitor(enabled=False)
>> +        self.vm.launch()
>> +        self.vm.wait()
>> +        self.assertEquals(self.vm.exitcode(), 1, "QEMU exit code should be 1")
>> +        self.assertRegex(self.vm.get_log(), r'phys-bits too low')
>> +
>> +    # now lets test some 64-bit CPU cases.
>> +    def test_phybits_low_tcg_q35_70_amd(self):
>> +        """
>> +        :avocado: tags=machine:q35
>> +        :avocado: tags=arch:x86_64
>> +
>> +        For q35 7.1 machines and above, "above_4G" memory starts at 1 TiB
>> +        boundary for AMD cpus (default). Lets test without that case.

^ first time I hear about that. Weird AMD-specific stuff, really.

     /*
      * The HyperTransport range close to the 1T boundary is unique to AMD
      * hosts with IOMMUs enabled. Restrict the ram-above-4g relocation
      * to above 1T to AMD vCPUs only. @enforce_amd_1tb_hole is only false in
      * older machine types (<= 7.0) for compatibility purposes.
      */
     if (IS_AMD_CPU(&cpu->env) && pcmc->enforce_amd_1tb_hole) {
         /* Bail out if max possible address does not cross HT range */
         if (pc_max_used_gpa(pcms, pci_hole64_size) >= AMD_HT_START) {
             x86ms->above_4g_mem_start = AMD_ABOVE_1TB_START;

So there is a fixed memory hole that starts at 0xfd00000000UL. If our max GPA would
overlap that region, we'll place everything above that memory hole.


         }

         /*
          * Advertise the HT region if address space covers the reserved
          * region or if we relocate.
          */
         if (cpu->phys_bits >= 40) {
             e820_add_entry(AMD_HT_START, AMD_HT_SIZE, E820_RESERVED);
         }
     }

I don't think your desciption is quite correct for that case. "above_4G" only starts above
1 TiB if it would overlap that special memory hole starting at 0xfd00000000UL.

> On 02-Nov-2023, at 2:13 PM, David Hildenbrand <david@redhat.com> wrote:
> 
> On 01.11.23 02:29, Ani Sinha wrote:
>>> On 27-Oct-2023, at 4:12 PM, Ani Sinha <anisinha@redhat.com> wrote:
>>> 
>>> QEMU has validations to make sure that a VM is not started with more memory
>>> (static and hotpluggable memory) than what the guest processor can address
>>> directly with its addressing bits. This change adds a test to make sure QEMU
>>> fails to start with a specific error message when an attempt is made to
>>> start a VM with more memory than what the processor can directly address.
>>> The test also checks for passing cases when the address space of the processor
>>> is capable of addressing all memory. Boundary cases are tested.
>>> 
>>> CC: imammedo@redhat.com
>>> CC: David Hildenbrand <david@redhat.com>
>>> Signed-off-by: Ani Sinha <anisinha@redhat.com>
>> Gentle ping on this. David does these tests look good and cover all cases more or less?
> 
> Hi,
> 
> sorry, for some reason the patches never made it to my inbox.
> 
>>> ---
>>> tests/avocado/mem-addr-space-check.py | 331 ++++++++++++++++++++++++++
>>> 1 file changed, 331 insertions(+)
>>> create mode 100644 tests/avocado/mem-addr-space-check.py
>>> 
>>> Changelog:
>>> v3: added pae tests as well.
>>> v2: added 64-bit tests. Added cxl tests.
>>> 
>>> Sample run:
>>> $ ./pyvenv/bin/avocado run tests/avocado/mem-addr-space-check.py --tap -
>>> 1..14
>>> ok 1 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_pse36
>>> ok 2 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_pae
>>> ok 3 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_pentium_pse36
>>> ok 4 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_pentium_pae
>>> ok 5 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_pentium2
>>> ok 6 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_nonpse36
>>> ok 7 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_tcg_q35_70_amd
>>> ok 8 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_tcg_q35_71_amd
>>> ok 9 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_tcg_q35_70_amd
>>> ok 10 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_tcg_q35_71_amd
>>> ok 11 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_tcg_q35_71_intel
>>> ok 12 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_tcg_q35_71_amd_41bits
>>> ok 13 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_low_tcg_q35_intel_cxl
>>> ok 14 tests/avocado/mem-addr-space-check.py:MemAddrCheck.test_phybits_ok_tcg_q35_intel_cxl
>>> 
>>> diff --git a/tests/avocado/mem-addr-space-check.py b/tests/avocado/mem-addr-space-check.py
>>> new file mode 100644
>>> index 0000000000..6ded11d658
>>> --- /dev/null
>>> +++ b/tests/avocado/mem-addr-space-check.py
>>> @@ -0,0 +1,331 @@
>>> +# Check for crash when using memory beyond the available guest processor
>>> +# address space.
>>> +#
>>> +# Copyright (c) 2023 Red Hat, Inc.
>>> +#
>>> +# Author:
>>> +#  Ani Sinha <anisinha@redhat.com>
>>> +#
>>> +# This program is free software; you can redistribute it and/or modify
>>> +# it under the terms of the GNU General Public License as published by
>>> +# the Free Software Foundation; either version 2 of the License, or
>>> +# (at your option) any later version.
>>> +#
>>> +# This program is distributed in the hope that it will be useful,
>>> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
>>> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>>> +# GNU General Public License for more details.
>>> +#
>>> +# You should have received a copy of the GNU General Public License
>>> +# along with this program.  If not, see <http://www.gnu.org/licenses/>.
>>> +
>>> +from avocado_qemu import QemuSystemTest
>>> +import signal
>>> +
>>> +class MemAddrCheck(QemuSystemTest):
>>> +    # first, lets test some 32-bit processors.
>>> +    # for all 32-bit cases, pci64_hole_size is 0.
>>> +    def test_phybits_low_pse36(self):
>>> +        """
>>> +        :avocado: tags=machine:q35
>>> +        :avocado: tags=arch:x86_64
>>> +
>>> +        With pse36 feature ON, a processor has 36 bits of addressing. So it can
>>> +        access up to a maximum of 64GiB of memory. Memory hotplug region begins
>>> +        at 4 GiB boundary when "above_4g_mem_size" is 0 (this would be true when
>>> +        we have 0.5 GiB of VM memory, see pc_q35_init()). This means total
>>> +        hotpluggable memory size is 60 GiB. Per slot, we reserve 1 GiB of memory
>>> +        for dimm alignment for all newer machines (see enforce_aligned_dimm
>>> +        property for pc machines and pc_get_device_memory_range()). That leaves
>>> +        total hotpluggable actual memory size of 59 GiB. If the VM is started
>>> +        with 0.5 GiB of memory, maxmem should be set to a maximum value of
>>> +        59.5 GiB to ensure that the processor can address all memory directly.
>>> +        Note that 64-bit pci hole size is 0 in this case. If maxmem is set to
>>> +        59.6G, QEMU should fail to start with a message "phy-bits are too low".
>>> +        If maxmem is set to 59.5G with all other QEMU parameters identical, QEMU
>>> +        should start fine.
>>> +        """
>>> +        self.vm.add_args('-S', '-machine', 'q35', '-m',
>>> +                         '512,slots=1,maxmem=59.6G',
>>> +                         '-cpu', 'pentium,pse36=on', '-display', 'none',
>>> +                         '-object', 'memory-backend-ram,id=mem1,size=1G',
>>> +                         '-device', 'virtio-mem-pci,id=vm0,memdev=mem1')
> 
> Just a note that for virtio-mem, you don't even have to reserve an (ACPI NVDIMM/DIMM) slot.
> So you might get away just setting "slots=0" and using maxmem=60.6G,

Ah ok I see https://lwn.net/Articles/755423/  :-) 

> making it possible to simplify the comment(s) a bit (and all the other test cases).
> 
> Otherwise, maybe clearer use a DIMM instead of a virtio-mem device.

I went with this approach in v4/v5.

> 
>>> +        self.vm.set_qmp_monitor(enabled=False)
>>> +        self.vm.launch()
>>> +        self.vm.wait()
>>> +        self.assertEquals(self.vm.exitcode(), 1, "QEMU exit code should be 1")
>>> +        self.assertRegex(self.vm.get_log(), r'phys-bits too low')
>>> +
>>> +    def test_phybits_low_pae(self):
>>> +        """
>>> +        :avocado: tags=machine:q35
>>> +        :avocado: tags=arch:x86_64
>>> +
>>> +        With pae feature ON, a processor has 36 bits of addressing. So it can
>>> +        access up to a maximum of 64GiB of memory. Rest is the same as the case
>>> +        with pse36 above.
>>> +        """
>>> +        self.vm.add_args('-S', '-machine', 'q35', '-m',
>>> +                         '512,slots=1,maxmem=59.6G',
>>> +                         '-cpu', 'pentium,pae=on', '-display', 'none',
>>> +                         '-object', 'memory-backend-ram,id=mem1,size=1G',
>>> +                         '-device', 'virtio-mem-pci,id=vm0,memdev=mem1')
> 
> Same considerations.
> 
>>> +        self.vm.set_qmp_monitor(enabled=False)
>>> +        self.vm.launch()
>>> +        self.vm.wait()
>>> +        self.assertEquals(self.vm.exitcode(), 1, "QEMU exit code should be 1")
>>> +        self.assertRegex(self.vm.get_log(), r'phys-bits too low')
>>> +
>>> +    def test_phybits_ok_pentium_pse36(self):
>>> +        """
>>> +        :avocado: tags=machine:q35
>>> +        :avocado: tags=arch:x86_64
>>> +
>>> +        Setting maxmem to 59.5G and making sure that QEMU can start with the
>>> +        same options as the failing case above with pse36 cpu feature.
>>> +        """
>>> +        self.vm.add_args('-machine', 'q35', '-m',
>>> +                         '512,slots=1,maxmem=59.5G',
>>> +                         '-cpu', 'pentium,pse36=on', '-display', 'none',
>>> +                         '-object', 'memory-backend-ram,id=mem1,size=1G',
>>> +                         '-device', 'virtio-mem-pci,id=vm0,memdev=mem1')
>>> +        self.vm.set_qmp_monitor(enabled=False)
>>> +        self.vm.launch()
>>> +        self.vm.shutdown()
>>> +        self.assertEquals(self.vm.exitcode(), -signal.SIGTERM,
>>> +                          "QEMU did not terminate gracefully upon SIGTERM")
>>> +
>>> +    def test_phybits_ok_pentium_pae(self):
>>> +        """
>>> +        :avocado: tags=machine:q35
>>> +        :avocado: tags=arch:x86_64
>>> +
>>> +        Test is same as above but now with pae CPUID turned on. Setting
> 
> s/CPUID/cpu feture/ ?
> 
>>> +        maxmem to 59.5G and making sure that QEMU can start fine with the
>>> +        same options as the case above.
>>> +        """
>>> +        self.vm.add_args('-machine', 'q35', '-m',
>>> +                         '512,slots=1,maxmem=59.5G',
>>> +                         '-cpu', 'pentium,pae=on', '-display', 'none',
>>> +                         '-object', 'memory-backend-ram,id=mem1,size=1G',
>>> +                         '-device', 'virtio-mem-pci,id=vm0,memdev=mem1')
>>> +        self.vm.set_qmp_monitor(enabled=False)
>>> +        self.vm.launch()
>>> +        self.vm.shutdown()
>>> +        self.assertEquals(self.vm.exitcode(), -signal.SIGTERM,
>>> +                          "QEMU did not terminate gracefully upon SIGTERM")
>>> +
>>> +    def test_phybits_ok_pentium2(self):
>>> +        """
>>> +        :avocado: tags=machine:q35
>>> +        :avocado: tags=arch:x86_64
>>> +
>>> +        Pentium2 has 36 bits of addressing, so its same as pentium
>>> +        with pse36 ON.
>>> +        """
>>> +        self.vm.add_args('-machine', 'q35', '-m',
>>> +                         '512,slots=1,maxmem=59.5G',
>>> +                         '-cpu', 'pentium2', '-display', 'none',
>>> +                         '-object', 'memory-backend-ram,id=mem1,size=1G',
>>> +                         '-device', 'virtio-mem-pci,id=vm0,memdev=mem1')
>>> +        self.vm.set_qmp_monitor(enabled=False)
>>> +        self.vm.launch()
>>> +        self.vm.shutdown()
>>> +        self.assertEquals(self.vm.exitcode(), -signal.SIGTERM,
>>> +                          "QEMU did not terminate gracefully upon SIGTERM")
>>> +
>>> +    def test_phybits_low_nonpse36(self):
>>> +        """
>>> +        :avocado: tags=machine:q35
>>> +        :avocado: tags=arch:x86_64
>>> +
>>> +        Pentium processor has 32 bits of addressing without pse36 or pae
>>> +        so it can access up to 4 GiB of memory. Setting maxmem to 4GiB
> 
> "access physical addresses up to 4 GiB"
> 
>>> +        should make QEMU fail to start with "phys-bits too low" message.
> 
> "because the region for memory hotplug is always placed above 4 GiB due to the
> PCI hole and simplicity."
> 
>>> +        """
>>> +        self.vm.add_args('-S', '-machine', 'q35', '-m',
>>> +                         '512,slots=1,maxmem=4G',
>>> +                         '-cpu', 'pentium', '-display', 'none',
>>> +                         '-object', 'memory-backend-ram,id=mem1,size=1G',
>>> +                         '-device', 'virtio-mem-pci,id=vm0,memdev=mem1')
>>> +        self.vm.set_qmp_monitor(enabled=False)
>>> +        self.vm.launch()
>>> +        self.vm.wait()
>>> +        self.assertEquals(self.vm.exitcode(), 1, "QEMU exit code should be 1")
>>> +        self.assertRegex(self.vm.get_log(), r'phys-bits too low')
>>> +
>>> +    # now lets test some 64-bit CPU cases.
>>> +    def test_phybits_low_tcg_q35_70_amd(self):
>>> +        """
>>> +        :avocado: tags=machine:q35
>>> +        :avocado: tags=arch:x86_64
>>> +
>>> +        For q35 7.1 machines and above, "above_4G" memory starts at 1 TiB
>>> +        boundary for AMD cpus (default). Lets test without that case.
> 
> ^ first time I hear about that. Weird AMD-specific stuff, really.
> 
>    /*
>     * The HyperTransport range close to the 1T boundary is unique to AMD
>     * hosts with IOMMUs enabled. Restrict the ram-above-4g relocation
>     * to above 1T to AMD vCPUs only. @enforce_amd_1tb_hole is only false in
>     * older machine types (<= 7.0) for compatibility purposes.
>     */
>    if (IS_AMD_CPU(&cpu->env) && pcmc->enforce_amd_1tb_hole) {
>        /* Bail out if max possible address does not cross HT range */
>        if (pc_max_used_gpa(pcms, pci_hole64_size) >= AMD_HT_START) {
>            x86ms->above_4g_mem_start = AMD_ABOVE_1TB_START;
> 
> So there is a fixed memory hole that starts at 0xfd00000000UL. If our max GPA would
> overlap that region, we'll place everything above that memory hole.
> 
> 
>        }
> 
>        /*
>         * Advertise the HT region if address space covers the reserved
>         * region or if we relocate.
>         */
>        if (cpu->phys_bits >= 40) {
>            e820_add_entry(AMD_HT_START, AMD_HT_SIZE, E820_RESERVED);
>        }
>    }
> 
> I don't think your desciption is quite correct for that case. "above_4G" only starts above
> 1 TiB if it would overlap that special memory hole starting at 0xfd00000000UL.
> 
> 
> -- 
> Cheers,
> 
> David / dhildenb