| Message ID | 20241101141111.104803-6-mdoucha@suse.cz |
|---|---|
| State | New |
| Headers | show |
| Series | Add LVM and NFS tests for file data integrity | expand |
Hi Martin, [ Cc Li and Cyril due tst_set_timeout vs. tst_loader.sh and tst_run_shell.c. ] > Add NFS test which checks data integrity of random writes into a file, > with both buffered and direct I/O. LGTM. Reviewed-by: Petr Vorel <pvorel@suse.cz> I'll try to have look on the core work (fsplough.c) on Monday. Hopefully Li or Cyril would have look into this as well. > The lower loop count is necessary because NFS has very large block size, > up to 256KB on x86_64. The new tests take ~50 minutes to complete in total > on my laptop. With the default loop count, the TCP tests would all time out. BTW I got timeout when testing nfs10.sh -v 4 -t tcp on Tumbleweed VM (2 CPU, 1.4 GB RAM, 2 GB SWAP) on my laptop: ... nfs10 1 TINFO: === Testing on ext4 === nfs10 1 TINFO: Formatting ext4 with opts='/dev/loop0' nfs10 1 TINFO: Mounting device: mount -t ext4 /dev/loop0 /var/tmp/LTP_nfs10.zWBiahjI48/mntpoint nfs10 1 TINFO: timeout per run is 0h 5m 0s => nfs10.sh runs only for 5 min. nfs10 1 TINFO: mount.nfs: (linux nfs-utils 2.6.3) nfs10 1 TINFO: setup NFSv4, socket type tcp nfs10 1 TINFO: Mounting /var/tmp/LTP_nfs10.zWBiahjI48/4/0 nfs10 1 TINFO: Mounting NFS: mount -v -t nfs -o proto=tcp,vers=4 10.0.0.2:/var/tmp/LTP_nfs10.zWBiahjI48/mntpoint/4/tcp /var/tmp/LTP_nfs10.zWBiahjI48/4/0 nfs10 1 TINFO: Testing buffered write, buffered read tst_tmpdir.c:316: TINFO: Using /var/tmp//LTP_fspL4p41F as tmpdir (btrfs filesystem) tst_test.c:1890: TINFO: LTP version: 20240930-49-g3facdd035 tst_test.c:1894: TINFO: Tested kernel: 6.12.0-rc4-1.gf83465d-default #1 SMP PREEMPT_DYNAMIC Sun Oct 20 22:44:57 UTC 2024 (f83465d) x86_64 tst_test.c:1725: TINFO: Timeout per run is 0h 00m 30s fsplough.c:186: TINFO: Block size: 262144 tst_test.c:1733: TINFO: Updating max runtime to 0h 08m 32s tst_test.c:1725: TINFO: Timeout per run is 0h 09m 02s => fsplough.c is updated to 9 min, but that does not help due nfs10.sh not having updated. Test timed out, sending SIGTERM! If you are running on slow machine, try exporting LTP_TIMEOUT_MUL > 1 Sending SIGKILL to test process... Test is still running... 10 Test is still running... 9 Test is still running... 8 Test is still running... 7 Test is still running... 6 Test is still running... 5 Test is still running... 4 Test is still running... 3 Test is still running... 2 Test is still running... 1 Test is still running, sending SIGKILL Killed The quickest way would be to use the same calculation (* $TST_CNT + 5% for spare time) in nfs10.sh and increase tst_set_timeout with. Or, I wonder if using tst_loader.sh and tst_run_shell.c would help to integrate these. But I'm not sure how easily could be nfs_lib.sh integrated, e.g. * isn't it too late, when it uses tst_net.sh (maybe this file would need to be integrated) * should use keep using TST_ALL_FILESYSTEMS=1 from tst_test.sh or configure via all_filesystems=1 via json? OT: Other tests which are considerable are these which use netstress.c (via tst_netload_compare). Kind regards, Petr
Hi! > Test timed out, sending SIGTERM! > If you are running on slow machine, try exporting LTP_TIMEOUT_MUL > 1 > Sending SIGKILL to test process... > Test is still running... 10 > Test is still running... 9 > Test is still running... 8 > Test is still running... 7 > Test is still running... 6 > Test is still running... 5 > Test is still running... 4 > Test is still running... 3 > Test is still running... 2 > Test is still running... 1 > Test is still running, sending SIGKILL > Killed Isn't the propblem here that the fsplough.c itself does not have .max_runtime set? I think that main problem here is that we have a LTP test that is being executed from another LTP test which means that we have two layers of everything including timeouts. > The quickest way would be to use the same calculation (* $TST_CNT + 5% for spare > time) in nfs10.sh and increase tst_set_timeout with. > > Or, I wonder if using tst_loader.sh and tst_run_shell.c would help to integrate > these. But I'm not sure how easily could be nfs_lib.sh integrated, e.g. > * isn't it too late, when it uses tst_net.sh (maybe this file would need to be > integrated) > * should use keep using TST_ALL_FILESYSTEMS=1 from tst_test.sh or configure > via all_filesystems=1 via json? I guess that the best solution would be to add NFS support into the tst_test.c as another filesystem. That way we would get much more coverate than we do now. I guess that it would be doable as long as the configuration on how to do that is passed to the test library somehow.
Hi! > > The quickest way would be to use the same calculation (* $TST_CNT + 5% for spare > > time) in nfs10.sh and increase tst_set_timeout with. > > > > Or, I wonder if using tst_loader.sh and tst_run_shell.c would help to integrate > > these. But I'm not sure how easily could be nfs_lib.sh integrated, e.g. > > * isn't it too late, when it uses tst_net.sh (maybe this file would need to be > > integrated) > > * should use keep using TST_ALL_FILESYSTEMS=1 from tst_test.sh or configure > > via all_filesystems=1 via json? > > I guess that the best solution would be to add NFS support into the > tst_test.c as another filesystem. That way we would get much more > coverate than we do now. I guess that it would be doable as long as the > configuration on how to do that is passed to the test library somehow. I'm looking at the nfs_lib.sh, we do have a lot there actually since it runs for all filesystems and we run the nfs tests for different nfs versions as well. This does not seem that would integrate well into the tst_test all_filesystems testing. I will think a bit how to integrate things better, because having LTP test inside LTP test will never work well.
Hi! > I will think a bit how to integrate things better, because having LTP > test inside LTP test will never work well. So one posibility is to use the new shell test library that integrates cleanly with C, then we can have a shell nfs test that runs for all filesystems, mounts the nfs and then runs the C fsplough itself. Minimal example how to do that is: From b48184d4fa1918afe6c84d0e691f8d46c986e92c Mon Sep 17 00:00:00 2001 From: Cyril Hrubis <chrubis@suse.cz> Date: Mon, 4 Nov 2024 14:29:24 +0100 Subject: [PATCH] shell lib: Add example how to run C child Signed-off-by: Cyril Hrubis <chrubis@suse.cz> --- testcases/lib/run_tests.sh | 2 +- testcases/lib/tests/.gitignore | 1 + testcases/lib/tests/shell_c_child.c | 16 ++++++++++++++ testcases/lib/tests/shell_loader_c_child.sh | 24 +++++++++++++++++++++ 4 files changed, 42 insertions(+), 1 deletion(-) create mode 100644 testcases/lib/tests/shell_c_child.c create mode 100755 testcases/lib/tests/shell_loader_c_child.sh diff --git a/testcases/lib/run_tests.sh b/testcases/lib/run_tests.sh index 8ea615f96..40d415e6c 100755 --- a/testcases/lib/run_tests.sh +++ b/testcases/lib/run_tests.sh @@ -14,7 +14,7 @@ for i in shell_loader.sh shell_loader_all_filesystems.sh shell_loader_no_metadat shell_loader_wrong_metadata.sh shell_loader_invalid_metadata.sh\ shell_loader_supported_archs.sh shell_loader_filesystems.sh\ shell_loader_tcnt.sh shell_loader_kconfigs.sh shell_loader_tags.sh \ - shell_loader_invalid_block.sh; do + shell_loader_invalid_block.sh shell_loader_c_child.sh; do echo echo "*** Running $i ***" echo diff --git a/testcases/lib/tests/.gitignore b/testcases/lib/tests/.gitignore index da967c4d6..e9e163d13 100644 --- a/testcases/lib/tests/.gitignore +++ b/testcases/lib/tests/.gitignore @@ -4,3 +4,4 @@ shell_test03 shell_test04 shell_test05 shell_test06 +shell_c_child diff --git a/testcases/lib/tests/shell_c_child.c b/testcases/lib/tests/shell_c_child.c new file mode 100644 index 000000000..fda5133a6 --- /dev/null +++ b/testcases/lib/tests/shell_c_child.c @@ -0,0 +1,16 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Shell test C child example. + */ + +#define TST_NO_DEFAULT_MAIN +#include "tst_test.h" + +int main(void) +{ + tst_reinit(); + + tst_res(TPASS, "C child works fine!"); + + return 0; +} diff --git a/testcases/lib/tests/shell_loader_c_child.sh b/testcases/lib/tests/shell_loader_c_child.sh new file mode 100755 index 000000000..d190be6e2 --- /dev/null +++ b/testcases/lib/tests/shell_loader_c_child.sh @@ -0,0 +1,24 @@ +#!/bin/sh +# +# --- +# doc +# +# [Description] +# +# This is an example how to run C child from shell. +# --- +# +# --- +# env +# { +# } +# --- + +. tst_loader.sh + +if [ -n "LTP_IPC_PATH" ]; then + tst_res TPASS "LTP_IPC_PATH=$LTP_IPC_PATH!" +fi + +tst_res TINFO "Running C child" +shell_c_child
> Hi! > > Test timed out, sending SIGTERM! > > If you are running on slow machine, try exporting LTP_TIMEOUT_MUL > 1 > > Sending SIGKILL to test process... > > Test is still running... 10 > > Test is still running... 9 > > Test is still running... 8 > > Test is still running... 7 > > Test is still running... 6 > > Test is still running... 5 > > Test is still running... 4 > > Test is still running... 3 > > Test is still running... 2 > > Test is still running... 1 > > Test is still running, sending SIGKILL > > Killed > Isn't the propblem here that the fsplough.c itself does not have > .max_runtime set? It has timeout adjusted in the setup function: tst_set_max_runtime(bufsize * loop_count / (8 * 1024 * 1024)); > I think that main problem here is that we have a LTP test that is being > executed from another LTP test which means that we have two layers of > everything including timeouts. Similarly netstress.c, which is also called from tst_net.sh. But these tests does not run on all filesystems. > > The quickest way would be to use the same calculation (* $TST_CNT + 5% for spare > > time) in nfs10.sh and increase tst_set_timeout with. > > Or, I wonder if using tst_loader.sh and tst_run_shell.c would help to integrate > > these. But I'm not sure how easily could be nfs_lib.sh integrated, e.g. > > * isn't it too late, when it uses tst_net.sh (maybe this file would need to be > > integrated) > > * should use keep using TST_ALL_FILESYSTEMS=1 from tst_test.sh or configure > > via all_filesystems=1 via json? > I guess that the best solution would be to add NFS support into the > tst_test.c as another filesystem. That way we would get much more > coverate than we do now. I guess that it would be doable as long as the > configuration on how to do that is passed to the test library somehow. Test coverage would really increase a lot. Even there are more NFS testsuites, IMHO it'd be useful to bring this to LTP - we test a different things then fstests, pynfs, cthon, .... But OTOH it does not have much sense to run NFS tests on a loop device which backing file is on tmpfs (the default setup on distros which use tmpfs on /tmp). It's a question, whether we should start using /var/tmp as a default for TMPDIR (if exists). LTP does cleanup of the functions, unless it fails. Specially NFS tests when they timeout, the cleanup is not done due mounted filesystem being used. Then /var/tmp gets filled up quickly and reboot does not do automatic cleanup. Adding NFS support would require setup code from nfs_lib.sh (e.g. exportfs, run mount binary or start using mount(2), doing various checks etc) to be run by tst_test.c, via another process via fork() and execvp(), e.g. with LTP's tst_cmd(). I'm not sure if minimal required subset should be put in some shell script, which would be executed by tst_test.c (maybe using tst_loader.sh to join shared memory to unify the counter) or run individual commands one by one. WDYT? This simpler version would allow to add various NFS to all_filesystems. I wonder if we want to support just single NFS version e.g. the newest NFSv4.2 or more. I suppose we would test only on TCP (UDP is deprecated, not enabled by default on newer kernels). But even the core of the NFS setup in nfs_mount() uses tst_rhost_run() from tst_net.sh. E.g. some parts of tst_net.sh would need to be rewritten into C code (or it would have to call tst_net.sh, if gluing with tst_loader.sh would work). My original idea to convert all NFS tests would basically mean to rewrite quite to use even more nfs_lib.sh and tst_net.sh. code (there is a support for setting NFS version and protocol and other things). Kind regards, Petr
> Hi! > > I'm looking at the nfs_lib.sh, we do have a lot there actually since it > > runs for all filesystems and we run the nfs tests for different nfs > > versions as well. This does not seem that would integrate well into the > > tst_test all_filesystems testing. Yes (found that as well when replied to your first email). I thought you want to integrate NFS just for a shake of other tests, which use .all_filesystems (e.g. some NFS to be added into fs_type_whitelist[], e.g. "nfsv4), which would of course require u specific setup. That would, of course did not cover all range of NFS versions + also using UDP, which is used for current NFS tests (including this one). > > I will think a bit how to integrate things better, because having LTP > > test inside LTP test will never work well. Reviewed-by: Petr Vorel <pvorel@suse.cz> Tested-by: Petr Vorel <pvorel@suse.cz> Looks usable + later on tests which use netstress.c could be converted to use it as well. Kind regards, Petr > So one posibility is to use the new shell test library that integrates > cleanly with C, then we can have a shell nfs test that runs for all > filesystems, mounts the nfs and then runs the C fsplough itself. > Minimal example how to do that is: > From b48184d4fa1918afe6c84d0e691f8d46c986e92c Mon Sep 17 00:00:00 2001 > From: Cyril Hrubis <chrubis@suse.cz> > Date: Mon, 4 Nov 2024 14:29:24 +0100 > Subject: [PATCH] shell lib: Add example how to run C child > Signed-off-by: Cyril Hrubis <chrubis@suse.cz> > --- > testcases/lib/run_tests.sh | 2 +- > testcases/lib/tests/.gitignore | 1 + > testcases/lib/tests/shell_c_child.c | 16 ++++++++++++++ > testcases/lib/tests/shell_loader_c_child.sh | 24 +++++++++++++++++++++ > 4 files changed, 42 insertions(+), 1 deletion(-) > create mode 100644 testcases/lib/tests/shell_c_child.c > create mode 100755 testcases/lib/tests/shell_loader_c_child.sh > diff --git a/testcases/lib/run_tests.sh b/testcases/lib/run_tests.sh > index 8ea615f96..40d415e6c 100755 > --- a/testcases/lib/run_tests.sh > +++ b/testcases/lib/run_tests.sh > @@ -14,7 +14,7 @@ for i in shell_loader.sh shell_loader_all_filesystems.sh shell_loader_no_metadat > shell_loader_wrong_metadata.sh shell_loader_invalid_metadata.sh\ > shell_loader_supported_archs.sh shell_loader_filesystems.sh\ > shell_loader_tcnt.sh shell_loader_kconfigs.sh shell_loader_tags.sh \ > - shell_loader_invalid_block.sh; do > + shell_loader_invalid_block.sh shell_loader_c_child.sh; do > echo > echo "*** Running $i ***" > echo > diff --git a/testcases/lib/tests/.gitignore b/testcases/lib/tests/.gitignore > index da967c4d6..e9e163d13 100644 > --- a/testcases/lib/tests/.gitignore > +++ b/testcases/lib/tests/.gitignore > @@ -4,3 +4,4 @@ shell_test03 > shell_test04 > shell_test05 > shell_test06 > +shell_c_child > diff --git a/testcases/lib/tests/shell_c_child.c b/testcases/lib/tests/shell_c_child.c > new file mode 100644 > index 000000000..fda5133a6 > --- /dev/null > +++ b/testcases/lib/tests/shell_c_child.c > @@ -0,0 +1,16 @@ > +// SPDX-License-Identifier: GPL-2.0-or-later > +/* > + * Shell test C child example. > + */ > + > +#define TST_NO_DEFAULT_MAIN > +#include "tst_test.h" > + > +int main(void) > +{ > + tst_reinit(); > + > + tst_res(TPASS, "C child works fine!"); > + > + return 0; > +} > diff --git a/testcases/lib/tests/shell_loader_c_child.sh b/testcases/lib/tests/shell_loader_c_child.sh > new file mode 100755 > index 000000000..d190be6e2 > --- /dev/null > +++ b/testcases/lib/tests/shell_loader_c_child.sh > @@ -0,0 +1,24 @@ > +#!/bin/sh > +# > +# --- > +# doc > +# > +# [Description] > +# > +# This is an example how to run C child from shell. > +# --- > +# > +# --- > +# env > +# { > +# } > +# --- > + > +. tst_loader.sh > + > +if [ -n "LTP_IPC_PATH" ]; then > + tst_res TPASS "LTP_IPC_PATH=$LTP_IPC_PATH!" > +fi > + > +tst_res TINFO "Running C child" > +shell_c_child
diff --git a/runtest/net.nfs b/runtest/net.nfs index 7f84457bc..fef993da8 100644 --- a/runtest/net.nfs +++ b/runtest/net.nfs @@ -94,6 +94,17 @@ nfs09_v40_ip6t nfs09.sh -6 -v 4 -t tcp nfs09_v41_ip6t nfs09.sh -6 -v 4.1 -t tcp nfs09_v42_ip6t nfs09.sh -6 -v 4.2 -t tcp +nfs10_v30_ip4u nfs10.sh -v 3 -t udp +nfs10_v30_ip4t nfs10.sh -v 3 -t tcp +nfs10_v40_ip4t nfs10.sh -v 4 -t tcp +nfs10_v41_ip4t nfs10.sh -v 4.1 -t tcp +nfs10_v42_ip4t nfs10.sh -v 4.2 -t tcp +nfs10_v30_ip6u nfs10.sh -6 -v 3 -t udp +nfs10_v30_ip6t nfs10.sh -6 -v 3 -t tcp +nfs10_v40_ip6t nfs10.sh -6 -v 4 -t tcp +nfs10_v41_ip6t nfs10.sh -6 -v 4.1 -t tcp +nfs10_v42_ip6t nfs10.sh -6 -v 4.2 -t tcp + nfslock01_v30_ip4u nfslock01.sh -v 3 -t udp nfslock01_v30_ip4t nfslock01.sh -v 3 -t tcp nfslock01_v40_ip4t nfslock01.sh -v 4 -t tcp diff --git a/testcases/network/nfs/nfs_stress/nfs10.sh b/testcases/network/nfs/nfs_stress/nfs10.sh new file mode 100755 index 000000000..b4e777285 --- /dev/null +++ b/testcases/network/nfs/nfs_stress/nfs10.sh @@ -0,0 +1,36 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (C) 2024 SUSE LLC <mdoucha@suse.cz> +# +# DESCRIPTION: Verify data integrity over NFS, with and without O_DIRECT + +TST_CNT=4 +TST_TESTFUNC="do_test" +TST_DEVICE_SIZE=1024 + +do_test1() +{ + tst_res TINFO "Testing buffered write, buffered read" + EXPECT_PASS fsplough -c 512 -d "$PWD" +} + +do_test2() +{ + tst_res TINFO "Testing buffered write, direct read" + EXPECT_PASS fsplough -c 512 -R -d "$PWD" +} + +do_test3() +{ + tst_res TINFO "Testing direct write, buffered read" + EXPECT_PASS fsplough -c 512 -W -d "$PWD" +} + +do_test4() +{ + tst_res TINFO "Testing direct write, direct read" + EXPECT_PASS fsplough -c 512 -RW -d "$PWD" +} + +. nfs_lib.sh +tst_run
Add NFS test which checks data integrity of random writes into a file, with both buffered and direct I/O. Signed-off-by: Martin Doucha <mdoucha@suse.cz> --- The lower loop count is necessary because NFS has very large block size, up to 256KB on x86_64. The new tests take ~50 minutes to complete in total on my laptop. With the default loop count, the TCP tests would all time out. runtest/net.nfs | 11 +++++++ testcases/network/nfs/nfs_stress/nfs10.sh | 36 +++++++++++++++++++++++ 2 files changed, 47 insertions(+) create mode 100755 testcases/network/nfs/nfs_stress/nfs10.sh