| Message ID | 20240903140326.773060-1-pvorel@suse.cz |
|---|---|
| State | Superseded |
| Headers | show |
| Series | [v2,1/1] swapoff01: Define max_runtime 45s | expand |
Hi! > Test run mostly below 1 sec per filesystem, only NTFS is very slow > ~ 10-15 sec depending on architecture. If part of the slowdown is caused by the fact that we format the filesystems the real question is if all the tests that format and mount a device should have a max_runtime defined. It may make sense to increment the max_runtime in the test library to compensate for the mkfs, but it looks that it's not the mkfs that is slow. I did some measurements for the swapoff01 case and the whole testrun runs for about 10 seconds for me. If I change the test to do tst_brk(TCONF, ...) as the first thing in the test setup its 1s. So writing out the 1MB of swapfile and running mkswap and swapon() on that takes about 90% of the runtime. So yes it looks like the actual test needs some runtime defined, although I'm not sure how long it should be. We probably need a reference machine to callibrate this on. I would avoid any emulated CPU and choose a real hardware. I guess one of the RPis e.g. Pi Zero would be a good choice since they are widely available and slow by any modern standards. > Signed-off-by: Petr Vorel <pvorel@suse.cz> > --- > Changes v1->v2: > * 60 => 45 > > testcases/kernel/syscalls/swapoff/swapoff01.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/testcases/kernel/syscalls/swapoff/swapoff01.c b/testcases/kernel/syscalls/swapoff/swapoff01.c > index 314630267d..ef75c92de0 100644 > --- a/testcases/kernel/syscalls/swapoff/swapoff01.c > +++ b/testcases/kernel/syscalls/swapoff/swapoff01.c > @@ -53,5 +53,6 @@ static struct tst_test test = { > .all_filesystems = 1, > .needs_root = 1, > .test_all = verify_swapoff, > + .max_runtime = 45, > .setup = setup > }; > -- > 2.45.2 >
> Hi! > > Test run mostly below 1 sec per filesystem, only NTFS is very slow > > ~ 10-15 sec depending on architecture. > If part of the slowdown is caused by the fact that we format the > filesystems the real question is if all the tests that format and mount > a device should have a max_runtime defined. It may make sense to > increment the max_runtime in the test library to compensate for the > mkfs, but it looks that it's not the mkfs that is slow. > I did some measurements for the swapoff01 case and the whole testrun > runs for about 10 seconds for me. If I change the test to do > tst_brk(TCONF, ...) as the first thing in the test setup its 1s. So > writing out the 1MB of swapfile and running mkswap and swapon() on that > takes about 90% of the runtime. Thanks for your investigation. > So yes it looks like the actual test needs some runtime defined, > although I'm not sure how long it should be. We probably need a > reference machine to callibrate this on. I would avoid any emulated CPU > and choose a real hardware. I guess one of the RPis e.g. Pi Zero would > be a good choice since they are widely available and slow by any modern > standards. This makes sense. Kind regards, Petr > > Signed-off-by: Petr Vorel <pvorel@suse.cz> > > --- > > Changes v1->v2: > > * 60 => 45 > > testcases/kernel/syscalls/swapoff/swapoff01.c | 1 + > > 1 file changed, 1 insertion(+) > > diff --git a/testcases/kernel/syscalls/swapoff/swapoff01.c b/testcases/kernel/syscalls/swapoff/swapoff01.c > > index 314630267d..ef75c92de0 100644 > > --- a/testcases/kernel/syscalls/swapoff/swapoff01.c > > +++ b/testcases/kernel/syscalls/swapoff/swapoff01.c > > @@ -53,5 +53,6 @@ static struct tst_test test = { > > .all_filesystems = 1, > > .needs_root = 1, > > .test_all = verify_swapoff, > > + .max_runtime = 45, > > .setup = setup > > }; > > -- > > 2.45.2
Hi! > > So yes it looks like the actual test needs some runtime defined, > > although I'm not sure how long it should be. We probably need a > > reference machine to callibrate this on. I would avoid any emulated CPU > > and choose a real hardware. I guess one of the RPis e.g. Pi Zero would > > be a good choice since they are widely available and slow by any modern > > standards. > > This makes sense. So I dusted off my Pi Zero and indeed the test timeouts there as well. The ntfs filesystem takes 78s, ext2 29s, exfat 20s. So I would say that max_runtime should be at least a minute. I guess I will try to run the whole run there before the release and check for timeouts so that we are sure that the runtime is set reasonably even for very slow hardware.
On Wed, Sep 4, 2024 at 4:53 PM Cyril Hrubis <chrubis@suse.cz> wrote: > > Hi! > > > So yes it looks like the actual test needs some runtime defined, > > > although I'm not sure how long it should be. We probably need a > > > reference machine to callibrate this on. I would avoid any emulated CPU > > > and choose a real hardware. I guess one of the RPis e.g. Pi Zero would > > > be a good choice since they are widely available and slow by any modern > > > standards. > > > > This makes sense. > > So I dusted off my Pi Zero and indeed the test timeouts there as well. > The ntfs filesystem takes 78s, ext2 29s, exfat 20s. So I would say that I'm wondering which distribution (and kernel version) did you use on raspi-0? My raspi4(ubuntu-24, 6.8.0-1010, Cortex-A72) takes a little faster. ext2: 12s ext3: 12s ext4: 0.4s ntfs: 46s exfat: skip > max_runtime should be at least a minute. I guess I will try to run the Agree. > whole run there before the release and check for timeouts so that we are > sure that the runtime is set reasonably even for very slow hardware. +1
On Wed, Sep 4, 2024 at 5:26 PM Li Wang <liwang@redhat.com> wrote: > > On Wed, Sep 4, 2024 at 4:53 PM Cyril Hrubis <chrubis@suse.cz> wrote: > > > > Hi! > > > > So yes it looks like the actual test needs some runtime defined, > > > > although I'm not sure how long it should be. We probably need a > > > > reference machine to callibrate this on. I would avoid any emulated CPU > > > > and choose a real hardware. I guess one of the RPis e.g. Pi Zero would > > > > be a good choice since they are widely available and slow by any modern > > > > standards. > > > > > > This makes sense. > > > > So I dusted off my Pi Zero and indeed the test timeouts there as well. > > The ntfs filesystem takes 78s, ext2 29s, exfat 20s. So I would say that > > I'm wondering which distribution (and kernel version) did you use on raspi-0? I ask this because we might need to consider the oldest supported systems: - https://linux-test-project.readthedocs.io/en/latest/users/supported_systems.html
> On Wed, Sep 4, 2024 at 5:26 PM Li Wang <liwang@redhat.com> wrote: > > On Wed, Sep 4, 2024 at 4:53 PM Cyril Hrubis <chrubis@suse.cz> wrote: > > > Hi! > > > > > So yes it looks like the actual test needs some runtime defined, > > > > > although I'm not sure how long it should be. We probably need a > > > > > reference machine to callibrate this on. I would avoid any emulated CPU > > > > > and choose a real hardware. I guess one of the RPis e.g. Pi Zero would > > > > > be a good choice since they are widely available and slow by any modern > > > > > standards. Pi Zero is good that it's slow enough that even emulated risc-v will work. But OTOH might be more practical to measure on RPI4 or RPI3 (maybe more of us has RPI4 or RPI3 personally or in CI). I mean unless others have/will buy Pi Zero you will be forced to do the reference. Also Pi Zero is slow (you need to compile, thus I will not love to do the measurements. Also emulated risc-v and Pi Zero will always need to set LTP_RUNTIME_MUL, right? Or do we want to do measurements that even these will run without LTP_RUNTIME_MUL? That would help users not having to bother, but increasing limits for fast devices might not be a good idea. > > > > This makes sense. > > > So I dusted off my Pi Zero and indeed the test timeouts there as well. > > > The ntfs filesystem takes 78s, ext2 29s, exfat 20s. So I would say that > > I'm wondering which distribution (and kernel version) did you use on raspi-0? Yes, 78s, ext2 29s, exfat 20s is faster, but that's due Raspberry Pi Zero using just Broadcom BCM2835 SoC as the first generation Raspberry Pi [1] (32 bit arm). RPI4 uses Broadcom BCM2711 SoC (64-bit quad-core). Kind regards, Petr [1] https://en.wikipedia.org/wiki/Raspberry_Pi#Processor > I ask this because we might need to consider the oldest supported systems: > - https://linux-test-project.readthedocs.io/en/latest/users/supported_systems.html
Hi! > I'm wondering which distribution (and kernel version) did you use on raspi-0? The default raspbian. The difference I guess is that RPi Zero is a single ARMv6 if I'm looking right it's the same CPU as on the original RPi. That's way slower than quad Cortex-A72 on RPi 4. > My raspi4(ubuntu-24, 6.8.0-1010, Cortex-A72) takes a little faster. > > ext2: 12s > ext3: 12s > ext4: 0.4s > ntfs: 46s > exfat: skip I suppose that this is stil slow on RPi 4 because the rootfs is on a SD card, since the CPU on RPi 4 is way faster than RPi Zero.
On Wed, Sep 4, 2024 at 10:00 PM Cyril Hrubis <chrubis@suse.cz> wrote: > Hi! > > I'm wondering which distribution (and kernel version) did you use on > raspi-0? > > The default raspbian. The difference I guess is that RPi Zero is a > single ARMv6 if I'm looking right it's the same CPU as on the original > RPi. That's way slower than quad Cortex-A72 on RPi 4. > > > My raspi4(ubuntu-24, 6.8.0-1010, Cortex-A72) takes a little faster. > > > > ext2: 12s > > ext3: 12s > > ext4: 0.4s > > ntfs: 46s > > exfat: skip > > I suppose that this is stil slow on RPi 4 because the rootfs is on a SD > card, since the CPU on RPi 4 is way faster than RPi Zero. > Yes, exactly. @Petr, if you planning to send out V3, I think it is better to set a proper value for max_runtime in swapon01 simultaneously. Since our CI jobs catch that one timeout out sporadically too. 7 tst_device.c:96: TINFO: Found free device 0 '/dev/loop0' 8 tst_test.c:1733: TINFO: LTP version: 20240524 9 tst_test.c:1617: TINFO: Timeout per run is 0h 01m 00s 10 tst_supported_fs_types.c:96: TINFO: Kernel supports ext2 11 tst_supported_fs_types.c:61: TINFO: mkfs.ext2 does exist 61 tst_test.c:1693: TINFO: === Testing on vfat === 62 tst_test.c:1106: TINFO: Formatting /dev/loop0 with vfat opts='' extra opts='' 63 tst_test.c:1120: TINFO: Mounting /dev/loop0 to /mnt/testarea/ltp-ukK7X0nhcs/LTP_swakCd462/mntpoint fstyp=vfat flags=0 64 libswap.c:198: TINFO: create a swapfile size of 1 megabytes (MB) 65 tst_ioctl.c:26: TINFO: FIBMAP ioctl is supported 66 swapon01.c:41: TINFO: create a swapfile size of 128 megabytes (MB) 67 swapon01.c:27: TPASS: tst_syscall(__NR_swapon, SWAP_FILE, 0) passed 68 tst_test.c:1671: TINFO: If you are running on slow machine, try exporting LTP_TIMEOUT_MUL > 1 69 tst_test.c:1673: TBROK: Test killed! (timeout?)
diff --git a/testcases/kernel/syscalls/swapoff/swapoff01.c b/testcases/kernel/syscalls/swapoff/swapoff01.c index 314630267d..ef75c92de0 100644 --- a/testcases/kernel/syscalls/swapoff/swapoff01.c +++ b/testcases/kernel/syscalls/swapoff/swapoff01.c @@ -53,5 +53,6 @@ static struct tst_test test = { .all_filesystems = 1, .needs_root = 1, .test_all = verify_swapoff, + .max_runtime = 45, .setup = setup };
Test run mostly below 1 sec per filesystem, only NTFS is very slow ~ 10-15 sec depending on architecture. Also this is the only tests in syscalls which fails on due timeout on emulated risc-v with LTP_TIMEOUT_MUL=2,LTP_RUNTIME_MUL=2 (with the default 30s runtime => 1 min due LTP_RUNTIME_MUL=2). Therefore set runtime to 45s, that should be enough even for slow risc-v and not that high for other archs. Signed-off-by: Petr Vorel <pvorel@suse.cz> --- Changes v1->v2: * 60 => 45 testcases/kernel/syscalls/swapoff/swapoff01.c | 1 + 1 file changed, 1 insertion(+)