| Message ID | 20241204172312.34099-2-mdoucha@suse.cz |
|---|---|
| State | Accepted |
| Headers | show |
| Series | [v3,1/2] nfs_lib.sh: Add support for extra NFS mount options | expand |
> Add NFS test which checks data integrity of random writes into a file, > with both buffered and direct I/O. > Signed-off-by: Martin Doucha <mdoucha@suse.cz> > --- > Changes since v1: Added TST_TIMEOUT > Changes since v2: Force NFS blocksize to the underlying FS blocksize > By default, NFS uses very large blocksize (up to 1MB) to optimize network > communication. For the FS integrity test this means a significant slowdown > because the size of the test file and I/O operations increase with > blocksize. Autodetect the blocksize of the underlying filesystem and force > the same value during NFS mount to speed things up. > The TST_TIMEOUT value is calculated as worst-case scenario for 64KB > blocksize. On reasonably modern x86_64 machines, the new tests take about > 6 minutes to complete in total (with Btrfs, Ext4 and XFS being tested). +1. I also did some testing. BTW TST_TIMEOUT is for single filesystem, thus 11 min is really safe enough even for slow machines like emulated risc-v. > +++ b/testcases/network/nfs/nfs_stress/nfs10.sh > @@ -0,0 +1,47 @@ > +#!/bin/sh > +# SPDX-License-Identifier: GPL-2.0-or-later > +# Copyright (C) 2024 SUSE LLC <mdoucha@suse.cz> > +# > +# DESCRIPTION: Verify data integrity over NFS, with and without O_DIRECT > + > +TST_CNT=4 > +TST_SETUP="nfs10_setup" > +TST_TESTFUNC="do_test" > +TST_DEVICE_SIZE=1024 > +TST_TIMEOUT=660 > + > +nfs10_setup() > +{ > + local bsize=$(stat -f -c %s .) > + > + bsize=$(( $bsize < 1024 ? 1024 : $bsize )) nit: this would work if stat is not installed (minimal systems) or fails for some reason: if [ -z "$bsize" ] || [ "$bsize" -lt 1024 ]; then bsize=1024 fi If you agree, I can change it during merge. Otherwise LGTM. Thank you! Reviewed-by: Petr Vorel <pvorel@suse.cz> Kind regards, Petr > + NFS_MOUNT_OPTS="rsize=$bsize,wsize=$bsize" > + nfs_setup
Hi, On 05. 12. 24 12:10, Petr Vorel wrote: >> + >> +nfs10_setup() >> +{ >> + local bsize=$(stat -f -c %s .) >> + >> + bsize=$(( $bsize < 1024 ? 1024 : $bsize )) > > nit: this would work if stat is not installed (minimal systems) or fails for > some reason: > > if [ -z "$bsize" ] || [ "$bsize" -lt 1024 ]; then > bsize=1024 > fi > > If you agree, I can change it during merge. > Otherwise LGTM. Please do, thank you.
Hi, > +++ b/testcases/network/nfs/nfs_stress/nfs10.sh > @@ -0,0 +1,47 @@ > +#!/bin/sh > +# SPDX-License-Identifier: GPL-2.0-or-later > +# Copyright (C) 2024 SUSE LLC <mdoucha@suse.cz> > +# > +# DESCRIPTION: Verify data integrity over NFS, with and without O_DIRECT > + > +TST_CNT=4 > +TST_SETUP="nfs10_setup" > +TST_TESTFUNC="do_test" > +TST_DEVICE_SIZE=1024 FYI requiring 1GB free space can break the test (even for UDP which should be mostly skipped with TCONF due UDP being disabled on mainline kernel). On UDP it runs quickly (< 10s), but on TCP it runs for whole timeout (~ 33 min). This is not a new limitation in shell API, it always existed. But recent commit 8532eb2c61 ("Add support for setting loop device size in shell tests") increased change of hitting the problem. I don't think it should block this patchset, I'll send a patch fixing the API soon. Kind regards, Petr # ./nfs10.sh -v 3 -t udp nfs10 1 TINFO: Running: nfs10.sh -v 3 -t udp nfs10 1 TINFO: 10.0.0.2/24 -- 10.0.0.1/24 nfs10 1 TINFO: fd00:1:1:1::2/64 -- fd00:1:1:1::1/64 nfs10 1 TINFO: Using /tmp/LTP_nfs10.XI7vLEjIkA as tmpdir (tmpfs filesystem) tst_device.c:299: TWARN: Failed to create test_dev.img: ENOSPC (28) Usage: tst_device acquire [size [filename]] tst_device release /path/to/device tst_device clear /path/to/device nfs10 1 TWARN: Failed to acquire device tst_supported_fs_types.c:169: TINFO: Skipping ext2 as requested by the test tst_supported_fs_types.c:169: TINFO: Skipping ext3 as requested by the test tst_supported_fs_types.c:97: TINFO: Kernel supports ext4 tst_supported_fs_types.c:62: TINFO: mkfs.ext4 does exist tst_supported_fs_types.c:97: TINFO: Kernel supports xfs tst_supported_fs_types.c:62: TINFO: mkfs.xfs does exist tst_supported_fs_types.c:97: TINFO: Kernel supports btrfs tst_supported_fs_types.c:62: TINFO: mkfs.btrfs does exist tst_supported_fs_types.c:97: TINFO: Kernel supports bcachefs tst_supported_fs_types.c:62: TINFO: mkfs.bcachefs does exist tst_supported_fs_types.c:169: TINFO: Skipping vfat as requested by the test tst_supported_fs_types.c:169: TINFO: Skipping exfat as requested by the test tst_supported_fs_types.c:169: TINFO: Skipping ntfs as requested by the test tst_supported_fs_types.c:169: TINFO: Skipping tmpfs as requested by the test nfs10 1 TINFO: === Testing on ext4 === tst_device.c:389: TWARN: Failed to clear 512k block on Usage: tst_device acquire [size [filename]] tst_device release /path/to/device tst_device clear /path/to/device ... > +TST_TIMEOUT=660 ...
Hi Martin, > > > + bsize=$(( $bsize < 1024 ? 1024 : $bsize )) > > nit: this would work if stat is not installed (minimal systems) or fails for > > some reason: > > if [ -z "$bsize" ] || [ "$bsize" -lt 1024 ]; then > > bsize=1024 > > fi > > If you agree, I can change it during merge. > > Otherwise LGTM. > Please do, thank you. The remaining two patches merged with agreed change. Thanks! I suppose you send a patch to modify vectorize_buffer() fsplough.c. Kind regards, Petr
diff --git a/runtest/net.nfs b/runtest/net.nfs index 7f84457bc..fef993da8 100644 --- a/runtest/net.nfs +++ b/runtest/net.nfs @@ -94,6 +94,17 @@ nfs09_v40_ip6t nfs09.sh -6 -v 4 -t tcp nfs09_v41_ip6t nfs09.sh -6 -v 4.1 -t tcp nfs09_v42_ip6t nfs09.sh -6 -v 4.2 -t tcp +nfs10_v30_ip4u nfs10.sh -v 3 -t udp +nfs10_v30_ip4t nfs10.sh -v 3 -t tcp +nfs10_v40_ip4t nfs10.sh -v 4 -t tcp +nfs10_v41_ip4t nfs10.sh -v 4.1 -t tcp +nfs10_v42_ip4t nfs10.sh -v 4.2 -t tcp +nfs10_v30_ip6u nfs10.sh -6 -v 3 -t udp +nfs10_v30_ip6t nfs10.sh -6 -v 3 -t tcp +nfs10_v40_ip6t nfs10.sh -6 -v 4 -t tcp +nfs10_v41_ip6t nfs10.sh -6 -v 4.1 -t tcp +nfs10_v42_ip6t nfs10.sh -6 -v 4.2 -t tcp + nfslock01_v30_ip4u nfslock01.sh -v 3 -t udp nfslock01_v30_ip4t nfslock01.sh -v 3 -t tcp nfslock01_v40_ip4t nfslock01.sh -v 4 -t tcp diff --git a/testcases/network/nfs/nfs_stress/nfs10.sh b/testcases/network/nfs/nfs_stress/nfs10.sh new file mode 100755 index 000000000..2fca4f916 --- /dev/null +++ b/testcases/network/nfs/nfs_stress/nfs10.sh @@ -0,0 +1,47 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (C) 2024 SUSE LLC <mdoucha@suse.cz> +# +# DESCRIPTION: Verify data integrity over NFS, with and without O_DIRECT + +TST_CNT=4 +TST_SETUP="nfs10_setup" +TST_TESTFUNC="do_test" +TST_DEVICE_SIZE=1024 +TST_TIMEOUT=660 + +nfs10_setup() +{ + local bsize=$(stat -f -c %s .) + + bsize=$(( $bsize < 1024 ? 1024 : $bsize )) + NFS_MOUNT_OPTS="rsize=$bsize,wsize=$bsize" + nfs_setup +} + +do_test1() +{ + tst_res TINFO "Testing buffered write, buffered read" + EXPECT_PASS fsplough -c 512 -d "$PWD" +} + +do_test2() +{ + tst_res TINFO "Testing buffered write, direct read" + EXPECT_PASS fsplough -c 512 -R -d "$PWD" +} + +do_test3() +{ + tst_res TINFO "Testing direct write, buffered read" + EXPECT_PASS fsplough -c 512 -W -d "$PWD" +} + +do_test4() +{ + tst_res TINFO "Testing direct write, direct read" + EXPECT_PASS fsplough -c 512 -RW -d "$PWD" +} + +. nfs_lib.sh +tst_run
Add NFS test which checks data integrity of random writes into a file, with both buffered and direct I/O. Signed-off-by: Martin Doucha <mdoucha@suse.cz> --- Changes since v1: Added TST_TIMEOUT Changes since v2: Force NFS blocksize to the underlying FS blocksize By default, NFS uses very large blocksize (up to 1MB) to optimize network communication. For the FS integrity test this means a significant slowdown because the size of the test file and I/O operations increase with blocksize. Autodetect the blocksize of the underlying filesystem and force the same value during NFS mount to speed things up. The TST_TIMEOUT value is calculated as worst-case scenario for 64KB blocksize. On reasonably modern x86_64 machines, the new tests take about 6 minutes to complete in total (with Btrfs, Ext4 and XFS being tested). runtest/net.nfs | 11 ++++++ testcases/network/nfs/nfs_stress/nfs10.sh | 47 +++++++++++++++++++++++ 2 files changed, 58 insertions(+) create mode 100755 testcases/network/nfs/nfs_stress/nfs10.sh