| Message ID | 20231211133223.695048-1-andrey.drobyshev@virtuozzo.com |
|---|---|
| State | New |
| Headers | show |
| Series | iotests: don't run tests requiring cached writes in '-nocache' mode | expand |
On 12/11/23 15:32, Andrey Drobyshev wrote: > There're tests whose logic implies running without O_DIRECT set, > otherwise they fail when running iotests in '-nocache' mode. For these > tests let's add _require_no_o_direct() helper which can be put in the > preabmle and which makes sure '-nocache' isn't set. Use it to skip > running the following tests: > > * 271: creates files with unaligned sizes, thus producing multiple > errors like: > > qemu-io: can't open device /path/to/t.qcow2.raw: Cannot get 'write' > permission without 'resize': Image size is not a multiple of request alignment > > * 308, file-io-error: use fuse exports. Though fuse does have > 'direct-io' mode (see https://docs.kernel.org/filesystems/fuse-io.html) > we aren't using it yet, thus getting errors like: > > qemu-io: can't open device /path/to/t.qcow2.fuse: Could not open > '/path/to/t.qcow2.fuse': filesystem does not support O_DIRECT > > Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com> > --- > tests/qemu-iotests/271 | 1 + > tests/qemu-iotests/308 | 2 ++ > tests/qemu-iotests/common.rc | 7 +++++++ > tests/qemu-iotests/tests/file-io-error | 1 + > 4 files changed, 11 insertions(+) > > diff --git a/tests/qemu-iotests/271 b/tests/qemu-iotests/271 > index 59a6fafa2f..1424b6954d 100755 > --- a/tests/qemu-iotests/271 > +++ b/tests/qemu-iotests/271 > @@ -44,6 +44,7 @@ _supported_fmt qcow2 > _supported_proto file nfs > _supported_os Linux > _unsupported_imgopts extended_l2 compat=0.10 cluster_size data_file refcount_bits=1[^0-9] > +_require_no_o_direct > > l2_offset=$((0x40000)) > > diff --git a/tests/qemu-iotests/308 b/tests/qemu-iotests/308 > index de12b2b1b9..535455e5b1 100755 > --- a/tests/qemu-iotests/308 > +++ b/tests/qemu-iotests/308 > @@ -52,6 +52,8 @@ _unsupported_fmt vpc > _supported_proto file # We create the FUSE export manually > _supported_os Linux # We need /dev/urandom > > +_require_no_o_direct > + > # $1: Export ID > # $2: Options (beyond the node-name and ID) > # $3: Expected return value (defaults to 'return') > diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc > index 95c12577dd..f61eae73b4 100644 > --- a/tests/qemu-iotests/common.rc > +++ b/tests/qemu-iotests/common.rc > @@ -857,6 +857,13 @@ _check_o_direct() > [[ "$out" != *"O_DIRECT"* ]] > } > > +_require_no_o_direct() > +{ > + if [ $CACHEMODE == "none" ] || [ $CACHEMODE == "directsync" ]; then > + _notrun "not suitable for cache mode: $CACHEMODE (implies O_DIRECT)" > + fi > +} > + > _require_o_direct() > { > if ! _check_o_direct; then > diff --git a/tests/qemu-iotests/tests/file-io-error b/tests/qemu-iotests/tests/file-io-error > index 88ee5f670c..2b8dc7f009 100755 > --- a/tests/qemu-iotests/tests/file-io-error > +++ b/tests/qemu-iotests/tests/file-io-error > @@ -40,6 +40,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 > # Format-agnostic (we do not use any), but we do test the file protocol > _supported_proto file > _require_drivers blkdebug null-co > +_require_no_o_direct > > if [ "$IMGOPTSSYNTAX" = "true" ]; then > # We need `$QEMU_IO -f file` to work; IMGOPTSSYNTAX uses --image-opts, Ping
On 1/11/24 14:53, Andrey Drobyshev wrote: > On 12/11/23 15:32, Andrey Drobyshev wrote: >> There're tests whose logic implies running without O_DIRECT set, >> otherwise they fail when running iotests in '-nocache' mode. For these >> tests let's add _require_no_o_direct() helper which can be put in the >> preabmle and which makes sure '-nocache' isn't set. Use it to skip >> running the following tests: >> >> * 271: creates files with unaligned sizes, thus producing multiple >> errors like: >> >> qemu-io: can't open device /path/to/t.qcow2.raw: Cannot get 'write' >> permission without 'resize': Image size is not a multiple of request alignment >> >> * 308, file-io-error: use fuse exports. Though fuse does have >> 'direct-io' mode (see https://docs.kernel.org/filesystems/fuse-io.html) >> we aren't using it yet, thus getting errors like: >> >> qemu-io: can't open device /path/to/t.qcow2.fuse: Could not open >> '/path/to/t.qcow2.fuse': filesystem does not support O_DIRECT >> >> Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com> >> --- >> tests/qemu-iotests/271 | 1 + >> tests/qemu-iotests/308 | 2 ++ >> tests/qemu-iotests/common.rc | 7 +++++++ >> tests/qemu-iotests/tests/file-io-error | 1 + >> 4 files changed, 11 insertions(+) >> >> diff --git a/tests/qemu-iotests/271 b/tests/qemu-iotests/271 >> index 59a6fafa2f..1424b6954d 100755 >> --- a/tests/qemu-iotests/271 >> +++ b/tests/qemu-iotests/271 >> @@ -44,6 +44,7 @@ _supported_fmt qcow2 >> _supported_proto file nfs >> _supported_os Linux >> _unsupported_imgopts extended_l2 compat=0.10 cluster_size data_file refcount_bits=1[^0-9] >> +_require_no_o_direct >> >> l2_offset=$((0x40000)) >> >> diff --git a/tests/qemu-iotests/308 b/tests/qemu-iotests/308 >> index de12b2b1b9..535455e5b1 100755 >> --- a/tests/qemu-iotests/308 >> +++ b/tests/qemu-iotests/308 >> @@ -52,6 +52,8 @@ _unsupported_fmt vpc >> _supported_proto file # We create the FUSE export manually >> _supported_os Linux # We need /dev/urandom >> >> +_require_no_o_direct >> + >> # $1: Export ID >> # $2: Options (beyond the node-name and ID) >> # $3: Expected return value (defaults to 'return') >> diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc >> index 95c12577dd..f61eae73b4 100644 >> --- a/tests/qemu-iotests/common.rc >> +++ b/tests/qemu-iotests/common.rc >> @@ -857,6 +857,13 @@ _check_o_direct() >> [[ "$out" != *"O_DIRECT"* ]] >> } >> >> +_require_no_o_direct() >> +{ >> + if [ $CACHEMODE == "none" ] || [ $CACHEMODE == "directsync" ]; then >> + _notrun "not suitable for cache mode: $CACHEMODE (implies O_DIRECT)" >> + fi >> +} >> + >> _require_o_direct() >> { >> if ! _check_o_direct; then >> diff --git a/tests/qemu-iotests/tests/file-io-error b/tests/qemu-iotests/tests/file-io-error >> index 88ee5f670c..2b8dc7f009 100755 >> --- a/tests/qemu-iotests/tests/file-io-error >> +++ b/tests/qemu-iotests/tests/file-io-error >> @@ -40,6 +40,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 >> # Format-agnostic (we do not use any), but we do test the file protocol >> _supported_proto file >> _require_drivers blkdebug null-co >> +_require_no_o_direct >> >> if [ "$IMGOPTSSYNTAX" = "true" ]; then >> # We need `$QEMU_IO -f file` to work; IMGOPTSSYNTAX uses --image-opts, > > Ping Ping
On Mon, Dec 11, 2023 at 03:32:23PM +0200, Andrey Drobyshev wrote: > There're tests whose logic implies running without O_DIRECT set, > otherwise they fail when running iotests in '-nocache' mode. For these > tests let's add _require_no_o_direct() helper which can be put in the > preabmle and which makes sure '-nocache' isn't set. Use it to skip preamble > running the following tests: > > * 271: creates files with unaligned sizes, thus producing multiple > errors like: > > qemu-io: can't open device /path/to/t.qcow2.raw: Cannot get 'write' > permission without 'resize': Image size is not a multiple of request alignment I wonder if we can instead tweak the test to use larger sizes such that all accesses ARE aligned even with O_DIRECT. /me goes and reads the test... # Note that the image size is not a multiple of the cluster size _reset_img 2083k Ah - we really DO have a test that depends on odd sizing; where changing it to be more nicely aligned will break other assumptions in the test. > > * 308, file-io-error: use fuse exports. Though fuse does have > 'direct-io' mode (see https://docs.kernel.org/filesystems/fuse-io.html) > we aren't using it yet, thus getting errors like: > > qemu-io: can't open device /path/to/t.qcow2.fuse: Could not open > '/path/to/t.qcow2.fuse': filesystem does not support O_DIRECT And I agree that this one is beyond our control, so adding skip support makes sense. > > Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com> > --- > tests/qemu-iotests/271 | 1 + > tests/qemu-iotests/308 | 2 ++ > tests/qemu-iotests/common.rc | 7 +++++++ > tests/qemu-iotests/tests/file-io-error | 1 + > 4 files changed, 11 insertions(+) Reviewed-by: Eric Blake <eblake@redhat.com>
Am 11.12.2023 um 14:32 hat Andrey Drobyshev geschrieben: > There're tests whose logic implies running without O_DIRECT set, > otherwise they fail when running iotests in '-nocache' mode. For these > tests let's add _require_no_o_direct() helper which can be put in the > preabmle and which makes sure '-nocache' isn't set. Use it to skip > running the following tests: > > * 271: creates files with unaligned sizes, thus producing multiple > errors like: > > qemu-io: can't open device /path/to/t.qcow2.raw: Cannot get 'write' > permission without 'resize': Image size is not a multiple of request alignment > > * 308, file-io-error: use fuse exports. Though fuse does have > 'direct-io' mode (see https://docs.kernel.org/filesystems/fuse-io.html) > we aren't using it yet, thus getting errors like: > > qemu-io: can't open device /path/to/t.qcow2.fuse: Could not open > '/path/to/t.qcow2.fuse': filesystem does not support O_DIRECT > > Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com> How are you running qemu-iotests to make these tests fail? I tried to reproduce, but they just pass for me: $ tests/qemu-iotests/check -qcow2 -nocache 271 308 file-io-error [...] 271 pass [11:20:50] [11:21:11] 21.1s (last: 20.4s) 308 pass [11:21:11] [11:21:14] 3.3s (last: 3.3s) file-io-error pass [11:21:14] [11:21:14] 0.3s (last: 0.3s) Passed all 3 iotests $ tests/qemu-iotests/check -raw -nocache 271 308 file-io-error 271 not run [11:21:20] [11:21:21] ... not suitable for this image format: raw 308 pass [11:21:21] [11:21:24] 3.8s (last: 2.8s) file-io-error pass [11:21:24] [11:21:25] 0.3s (last: 0.3s) Not run: 271 Passed all 2 iotests Kevin
On 1/26/24 12:24, Kevin Wolf wrote: > Am 11.12.2023 um 14:32 hat Andrey Drobyshev geschrieben: >> There're tests whose logic implies running without O_DIRECT set, >> otherwise they fail when running iotests in '-nocache' mode. For these >> tests let's add _require_no_o_direct() helper which can be put in the >> preabmle and which makes sure '-nocache' isn't set. Use it to skip >> running the following tests: >> >> * 271: creates files with unaligned sizes, thus producing multiple >> errors like: >> >> qemu-io: can't open device /path/to/t.qcow2.raw: Cannot get 'write' >> permission without 'resize': Image size is not a multiple of request alignment >> >> * 308, file-io-error: use fuse exports. Though fuse does have >> 'direct-io' mode (see https://docs.kernel.org/filesystems/fuse-io.html) >> we aren't using it yet, thus getting errors like: >> >> qemu-io: can't open device /path/to/t.qcow2.fuse: Could not open >> '/path/to/t.qcow2.fuse': filesystem does not support O_DIRECT >> >> Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com> > > How are you running qemu-iotests to make these tests fail? I tried to > reproduce, but they just pass for me: > > $ tests/qemu-iotests/check -qcow2 -nocache 271 308 file-io-error > [...] > 271 pass [11:20:50] [11:21:11] 21.1s (last: 20.4s) > 308 pass [11:21:11] [11:21:14] 3.3s (last: 3.3s) > file-io-error pass [11:21:14] [11:21:14] 0.3s (last: 0.3s) > Passed all 3 iotests > > $ tests/qemu-iotests/check -raw -nocache 271 308 file-io-error > 271 not run [11:21:20] [11:21:21] ... not suitable for this image format: raw > 308 pass [11:21:21] [11:21:24] 3.8s (last: 2.8s) > file-io-error pass [11:21:24] [11:21:25] 0.3s (last: 0.3s) > Not run: 271 > Passed all 2 iotests > > Kevin > As for the test 271, I imagine this might be caused by different request alignment. The failure occurs in block.c, bdrv_node_refresh_perm(). If I print the alignment out explicitly, I get: qemu-io: can't open device /path/to/t.qcow2.raw: Cannot get 'write' permission without 'resize': Image size is not a multiple of request alignment: 4096 For the record, I'm running tests on ext4. I'm not sure about the fuse tests though. Could it also have smth to do with the underlying fs? Andrey
Am 26.01.2024 um 12:04 hat Andrey Drobyshev geschrieben: > On 1/26/24 12:24, Kevin Wolf wrote: > > Am 11.12.2023 um 14:32 hat Andrey Drobyshev geschrieben: > >> There're tests whose logic implies running without O_DIRECT set, > >> otherwise they fail when running iotests in '-nocache' mode. For these > >> tests let's add _require_no_o_direct() helper which can be put in the > >> preabmle and which makes sure '-nocache' isn't set. Use it to skip > >> running the following tests: > >> > >> * 271: creates files with unaligned sizes, thus producing multiple > >> errors like: > >> > >> qemu-io: can't open device /path/to/t.qcow2.raw: Cannot get 'write' > >> permission without 'resize': Image size is not a multiple of request alignment > >> > >> * 308, file-io-error: use fuse exports. Though fuse does have > >> 'direct-io' mode (see https://docs.kernel.org/filesystems/fuse-io.html) > >> we aren't using it yet, thus getting errors like: > >> > >> qemu-io: can't open device /path/to/t.qcow2.fuse: Could not open > >> '/path/to/t.qcow2.fuse': filesystem does not support O_DIRECT > >> > >> Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com> > > > > How are you running qemu-iotests to make these tests fail? I tried to > > reproduce, but they just pass for me: > > > > $ tests/qemu-iotests/check -qcow2 -nocache 271 308 file-io-error > > [...] > > 271 pass [11:20:50] [11:21:11] 21.1s (last: 20.4s) > > 308 pass [11:21:11] [11:21:14] 3.3s (last: 3.3s) > > file-io-error pass [11:21:14] [11:21:14] 0.3s (last: 0.3s) > > Passed all 3 iotests > > > > $ tests/qemu-iotests/check -raw -nocache 271 308 file-io-error > > 271 not run [11:21:20] [11:21:21] ... not suitable for this image format: raw > > 308 pass [11:21:21] [11:21:24] 3.8s (last: 2.8s) > > file-io-error pass [11:21:24] [11:21:25] 0.3s (last: 0.3s) > > Not run: 271 > > Passed all 2 iotests > > > > Kevin > > > > As for the test 271, I imagine this might be caused by different request > alignment. The failure occurs in block.c, bdrv_node_refresh_perm(). If > I print the alignment out explicitly, I get: > > qemu-io: can't open device /path/to/t.qcow2.raw: Cannot get 'write' > permission without 'resize': Image size is not a multiple of request > alignment: 4096 Oh, do you have a 4k sector size? I think for me O_DIRECT only enforces 512 byte alignment. I can later try again on a 4k sector size loopback device or something. If so, I think that's worth mentioning in the commit message. > For the record, I'm running tests on ext4. I don't expect that this makes the difference, but xfs for me. > I'm not sure about the fuse tests though. Could it also have smth to do > with the underlying fs? Probably the same request alignment thing then. Kevin
diff --git a/tests/qemu-iotests/271 b/tests/qemu-iotests/271 index 59a6fafa2f..1424b6954d 100755 --- a/tests/qemu-iotests/271 +++ b/tests/qemu-iotests/271 @@ -44,6 +44,7 @@ _supported_fmt qcow2 _supported_proto file nfs _supported_os Linux _unsupported_imgopts extended_l2 compat=0.10 cluster_size data_file refcount_bits=1[^0-9] +_require_no_o_direct l2_offset=$((0x40000)) diff --git a/tests/qemu-iotests/308 b/tests/qemu-iotests/308 index de12b2b1b9..535455e5b1 100755 --- a/tests/qemu-iotests/308 +++ b/tests/qemu-iotests/308 @@ -52,6 +52,8 @@ _unsupported_fmt vpc _supported_proto file # We create the FUSE export manually _supported_os Linux # We need /dev/urandom +_require_no_o_direct + # $1: Export ID # $2: Options (beyond the node-name and ID) # $3: Expected return value (defaults to 'return') diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc index 95c12577dd..f61eae73b4 100644 --- a/tests/qemu-iotests/common.rc +++ b/tests/qemu-iotests/common.rc @@ -857,6 +857,13 @@ _check_o_direct() [[ "$out" != *"O_DIRECT"* ]] } +_require_no_o_direct() +{ + if [ $CACHEMODE == "none" ] || [ $CACHEMODE == "directsync" ]; then + _notrun "not suitable for cache mode: $CACHEMODE (implies O_DIRECT)" + fi +} + _require_o_direct() { if ! _check_o_direct; then diff --git a/tests/qemu-iotests/tests/file-io-error b/tests/qemu-iotests/tests/file-io-error index 88ee5f670c..2b8dc7f009 100755 --- a/tests/qemu-iotests/tests/file-io-error +++ b/tests/qemu-iotests/tests/file-io-error @@ -40,6 +40,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # Format-agnostic (we do not use any), but we do test the file protocol _supported_proto file _require_drivers blkdebug null-co +_require_no_o_direct if [ "$IMGOPTSSYNTAX" = "true" ]; then # We need `$QEMU_IO -f file` to work; IMGOPTSSYNTAX uses --image-opts,
There're tests whose logic implies running without O_DIRECT set, otherwise they fail when running iotests in '-nocache' mode. For these tests let's add _require_no_o_direct() helper which can be put in the preabmle and which makes sure '-nocache' isn't set. Use it to skip running the following tests: * 271: creates files with unaligned sizes, thus producing multiple errors like: qemu-io: can't open device /path/to/t.qcow2.raw: Cannot get 'write' permission without 'resize': Image size is not a multiple of request alignment * 308, file-io-error: use fuse exports. Though fuse does have 'direct-io' mode (see https://docs.kernel.org/filesystems/fuse-io.html) we aren't using it yet, thus getting errors like: qemu-io: can't open device /path/to/t.qcow2.fuse: Could not open '/path/to/t.qcow2.fuse': filesystem does not support O_DIRECT Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com> --- tests/qemu-iotests/271 | 1 + tests/qemu-iotests/308 | 2 ++ tests/qemu-iotests/common.rc | 7 +++++++ tests/qemu-iotests/tests/file-io-error | 1 + 4 files changed, 11 insertions(+)