[1/2] support/testing: use aarch64 for TestPythonPy3Botocore

With new python-botocore versions, the size of the package will
increase and the 256MiB memory limit from the default versatilepb
machine will no longer be sufficient to do runtime tests.

Switch to using an aarch64 machine, and use 512MiB memory.

The configuration and login method are taken from TestOpenJdk.

While at it, add the python-botocore test files to the DEVELOPERS
file.

Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>
---
 DEVELOPERS                                    |  2 ++
 .../tests/package/test_python_botocore.py     | 24 ++++++++++++++++++-
 2 files changed, 25 insertions(+), 1 deletion(-)

Hello Raphaël,

On Wed, 15 Jun 2022 19:12:15 +0200
Raphaël Mélotte <raphael.melotte@mind.be> wrote:

> With new python-botocore versions, the size of the package will
> increase and the 256MiB memory limit from the default versatilepb
> machine will no longer be sufficient to do runtime tests.
> 
> Switch to using an aarch64 machine, and use 512MiB memory.
> 
> The configuration and login method are taken from TestOpenJdk.
> 
> While at it, add the python-botocore test files to the DEVELOPERS
> file.
> 
> Signed-off-by: Raphaël Mélotte <raphael.melotte@mind.be>

Thanks a lot for the proposal. I understand the need for more memory,
but I'm wondering if building a full aarch64 kernel just for this is
really the right solution.

What about instead trying to pass a custom option to Qemu?

support/testing/infra/emulator.py goes like this:

    def boot(self, arch, kernel=None, kernel_cmdline=None, options=None):
        if arch in ["armv7", "armv5"]:
            qemu_arch = "arm"
        else:
            qemu_arch = arch

        qemu_cmd = ["qemu-system-{}".format(qemu_arch),
                    "-serial", "stdio",
                    "-display", "none",
                    "-m", "256"]

        if options:
            qemu_cmd += options

So if we passe options=["-m", "512"], this should override the default
-m 256.

Then the question is how can the Python test infrastructure pass this
down to self.emulator.boot().

Something like this perhaps:

diff --git a/support/testing/tests/package/test_python.py b/support/testing/tests/package/test_python.py
index f29aff6624..bd0b21f193 100644
--- a/support/testing/tests/package/test_python.py
+++ b/support/testing/tests/package/test_python.py
@@ -11,11 +11,14 @@ class TestPythonBase(infra.basetest.BRTest):
         """
     interpreter = "python"
 
-    def login(self):
+    def login(self, qemu_extra_opts=None):
         cpio_file = os.path.join(self.builddir, "images", "rootfs.cpio")
+        qemu_opts = ["-initrd", cpio_file]
+        if qemu_extra_opts:
+            qemu_opts += qemu_extra_opts
         self.emulator.boot(arch="armv5",
                            kernel="builtin",
-                           options=["-initrd", cpio_file])
+                           options=qemu_opts)
         self.emulator.login()
 
     def version_test(self, version, timeout=-1):
@@ -80,6 +83,7 @@ class TestPythonPackageBase(TestPythonBase):
                    "{sample_scripts}")
     sample_scripts = None
     timeout = -1
+    qemu_extra_opts = None
 
     def __init__(self, names):
         """Add the scripts to the target in build time."""
@@ -102,6 +106,6 @@ class TestPythonPackageBase(TestPythonBase):
             self.assertRunOk(cmd, timeout=self.timeout)
 
     def test_run(self):
-        self.login()
+        self.login(qemu_extra_opts=self.qemu_extra_opts)
         self.check_sample_scripts_exist()
         self.run_sample_scripts()
diff --git a/support/testing/tests/package/test_python_botocore.py b/support/testing/tests/package/test_python_botocore.py
index 6336c5658f..a7c35a6ed6 100644
--- a/support/testing/tests/package/test_python_botocore.py
+++ b/support/testing/tests/package/test_python_botocore.py
@@ -10,3 +10,4 @@ class TestPythonPy3Botocore(TestPythonPackageBase):
         """
     sample_scripts = ["tests/package/sample_python_botocore.py"]
     timeout = 10
+    qemu_extra_opts = ["-m", "512"]

(Completely untested, and also I'm not sure if it's the most Pythonic
way to do that, I'm not a Python developer).

What do you think?

Best regards,

Thomas

Hello Thomas,

On 7/20/22 23:50, Thomas Petazzoni wrote:
> 
> Thanks a lot for the proposal.

Thanks for looking at it!

> I understand the need for more memory,
> but I'm wondering if building a full aarch64 kernel just for this is
> really the right solution.
> 
> What about instead trying to pass a custom option to Qemu?
> 
> support/testing/infra/emulator.py goes like this:
> 
>      def boot(self, arch, kernel=None, kernel_cmdline=None, options=None):
>          if arch in ["armv7", "armv5"]:
>              qemu_arch = "arm"
>          else:
>              qemu_arch = arch
> 
>          qemu_cmd = ["qemu-system-{}".format(qemu_arch),
>                      "-serial", "stdio",
>                      "-display", "none",
>                      "-m", "256"]
> 
>          if options:
>              qemu_cmd += options
> 
> So if we passe options=["-m", "512"], this should override the default
> -m 256.

That's what I tried at first, but the problem is that qemu refuses to use more than 256MiB for the versatilepb machine (I should have made that clearer in the commit message, sorry).

This is the place in qemu that checks that no more than 256MiB is used:
===
     if (machine->ram_size > 0x10000000) {
         /* Device starting at address 0x10000000,
          * and memory cannot overlap with devices.
          * Refuse to run rather than behaving very confusingly.
          */
         error_report("versatilepb: memory size must not exceed 256MB");
         exit(1);
     }
====

So I had no another choice than to use a different one.
I could use a different board than "aarch64-virt" if needed, I only chose it because it was already used elsewhere.

What do you think?

Kind regards,

Raphaël