| Message ID | 20220712021345.8530-6-faithilikerun@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | Add support for zoned device | expand |
On Mon, 11 Jul 2022 at 22:21, Sam Li <faithilikerun@gmail.com> wrote: > > We have added new block layer APIs of zoned block devices. Test it with: > (1) Create a null_blk device, run each zone operation on it and see > whether reporting right zone information. > > Signed-off-by: Sam Li <faithilikerun@gmail.com> > --- > tests/qemu-iotests/tests/zoned.sh | 69 +++++++++++++++++++++++++++++++ > 1 file changed, 69 insertions(+) > create mode 100755 tests/qemu-iotests/tests/zoned.sh > > diff --git a/tests/qemu-iotests/tests/zoned.sh b/tests/qemu-iotests/tests/zoned.sh > new file mode 100755 > index 0000000000..e14a3a420e > --- /dev/null > +++ b/tests/qemu-iotests/tests/zoned.sh > @@ -0,0 +1,69 @@ > +#!/usr/bin/env bash > +# > +# Test zone management operations. > +# > + > +seq="$(basename $0)" > +echo "QA output created by $seq" > +status=1 # failure is the default! > + > +_cleanup() > +{ > + _cleanup_test_img > + sudo rmmod null_blk > +} > +trap "_cleanup; exit \$status" 0 1 2 3 15 > + > +# get standard environment, filters and checks > +. ./common.rc > +. ./common.filter > +. ./common.qemu > + > +# This test only runs on Linux hosts with raw image files. > +_supported_fmt raw > +_supported_proto file > +_supported_os Linux > + > +QEMU_IO="build/qemu-io" > +IMG="--image-opts driver=zoned_host_device,filename=/dev/nullb0" > +QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT > + > +echo "Testing a null_blk device" > +echo "Simple cases: if the operations work" > +sudo modprobe null_blk nr_devices=1 zoned=1 Jens & Ming Lei: null_blk is not ideal for automated test cases because it requires root and is global. If two tests are run at the same time they will interfere with each other. There is no way to have independent instances of the null_blk kernel module and the /dev/nullb0 device on a single Linux system. I think this test case can be merged for now but if there's time I suggest investigating alternatives. Maybe the new Linux ublk_drv driver is a better choice if it supports unprivileged operation with multiple independent block devices (plus zoned storage!). It would be necessary to write a userspace null block device that supports zoned storage if that doesn't exist already. I have CCed Ming Lei and Jens Axboe for thoughts. > +# hidden issues: > +# 1. memory allocation error of "unaligned tcache chunk detected" when the nr_zone=1 in zone report > +# 2. qemu-io: after report 10 zones, the program failed at double free error and exited. What is this? It looks like a TODO list of bugs you hit? Have they already been solved? > +echo "report the first zone" > +sudo $QEMU_IO $IMG -c "zr 0 0 1" > +echo "report: the first 10 zones" > +sudo $QEMU_IO $IMG -c "zr 0 0 10" > + > +echo "open the first zone" > +sudo $QEMU_IO $IMG -c "zo 0 0x80000" > +echo "report after:" > +sudo $QEMU_IO $IMG -c "zr 0 0 1" > +echo "open the last zone" > +sudo $QEMU_IO $IMG -c "zo 0x3e70000000 0x80000" > +echo "report after:" > +sudo $QEMU_IO $IMG -c "zr 0x3e70000000 0 2" > + > +echo "close the first zone" > +sudo $QEMU_IO $IMG -c "zc 0 0x80000" > +echo "report after:" > +sudo $QEMU_IO $IMG -c "zr 0 0 1" > +echo "close the last zone" > +sudo $QEMU_IO $IMG -c "zc 0x3e70000000 0x80000" > +echo "report after:" > +sudo $QEMU_IO $IMG -c "zr 0x3e70000000 0 2" > + > + > +echo "reset the second zone" > +sudo $QEMU_IO $IMG -c "zrs 0x80000 0x80000" > +echo "After resetting a zone:" > +sudo $QEMU_IO $IMG -c "zr 0x80000 0 5" > + > +# success, all done > +echo "*** done" > +rm -f $seq.full > +status=0 This patch is missing the .out reference file. Did you forget to git-add(1) it?
On Wed, Jul 27, 2022 at 10:34:56AM -0400, Stefan Hajnoczi wrote: > On Mon, 11 Jul 2022 at 22:21, Sam Li <faithilikerun@gmail.com> wrote: > > > > We have added new block layer APIs of zoned block devices. Test it with: > > (1) Create a null_blk device, run each zone operation on it and see > > whether reporting right zone information. > > > > Signed-off-by: Sam Li <faithilikerun@gmail.com> > > --- > > tests/qemu-iotests/tests/zoned.sh | 69 +++++++++++++++++++++++++++++++ > > 1 file changed, 69 insertions(+) > > create mode 100755 tests/qemu-iotests/tests/zoned.sh > > > > diff --git a/tests/qemu-iotests/tests/zoned.sh b/tests/qemu-iotests/tests/zoned.sh > > new file mode 100755 > > index 0000000000..e14a3a420e > > --- /dev/null > > +++ b/tests/qemu-iotests/tests/zoned.sh > > @@ -0,0 +1,69 @@ > > +#!/usr/bin/env bash > > +# > > +# Test zone management operations. > > +# > > + > > +seq="$(basename $0)" > > +echo "QA output created by $seq" > > +status=1 # failure is the default! > > + > > +_cleanup() > > +{ > > + _cleanup_test_img > > + sudo rmmod null_blk > > +} > > +trap "_cleanup; exit \$status" 0 1 2 3 15 > > + > > +# get standard environment, filters and checks > > +. ./common.rc > > +. ./common.filter > > +. ./common.qemu > > + > > +# This test only runs on Linux hosts with raw image files. > > +_supported_fmt raw > > +_supported_proto file > > +_supported_os Linux > > + > > +QEMU_IO="build/qemu-io" > > +IMG="--image-opts driver=zoned_host_device,filename=/dev/nullb0" > > +QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT > > + > > +echo "Testing a null_blk device" > > +echo "Simple cases: if the operations work" > > +sudo modprobe null_blk nr_devices=1 zoned=1 > > Jens & Ming Lei: > > null_blk is not ideal for automated test cases because it requires > root and is global. If two tests are run at the same time they will > interfere with each other. There is no way to have independent > instances of the null_blk kernel module and the /dev/nullb0 device on > a single Linux system. I think this test case can be merged for now > but if there's time I suggest investigating alternatives. > > Maybe the new Linux ublk_drv driver is a better choice if it supports > unprivileged operation with multiple independent block devices (plus This can be one big topic, and IMO, the io path needs to be reviewed wrt. security risk. Also if one block device is stuck, it shouldn't affect others, so far it can be one problem at least in sync/writeback, if one device is hang, writeback on all block device may not move on. > zoned storage!). It would be necessary to write a userspace null block > device that supports zoned storage if that doesn't exist already. I > have CCed Ming Lei and Jens Axboe for thoughts. IMO, it shouldn't be hard to simulate zoned from userspace, the patches[1] for setting device parameters are just sent out, then zoned parameter could be sent to driver with the added commands easily. Even ublk can be used to implement zoned target, such as, ublk is exposed as one normal disk, but the backing disk could be one real zoned/zns device. [1] https://lore.kernel.org/linux-block/20220727141628.985429-1-ming.lei@redhat.com/T/#mb5518cf418b63fb6ca649f0df199137e50907a29 thanks, Ming
On Wed, 27 Jul 2022 at 11:00, Ming Lei <ming.lei@redhat.com> wrote: > > On Wed, Jul 27, 2022 at 10:34:56AM -0400, Stefan Hajnoczi wrote: > > On Mon, 11 Jul 2022 at 22:21, Sam Li <faithilikerun@gmail.com> wrote: > > > > > > We have added new block layer APIs of zoned block devices. Test it with: > > > (1) Create a null_blk device, run each zone operation on it and see > > > whether reporting right zone information. > > > > > > Signed-off-by: Sam Li <faithilikerun@gmail.com> > > > --- > > > tests/qemu-iotests/tests/zoned.sh | 69 +++++++++++++++++++++++++++++++ > > > 1 file changed, 69 insertions(+) > > > create mode 100755 tests/qemu-iotests/tests/zoned.sh > > > > > > diff --git a/tests/qemu-iotests/tests/zoned.sh b/tests/qemu-iotests/tests/zoned.sh > > > new file mode 100755 > > > index 0000000000..e14a3a420e > > > --- /dev/null > > > +++ b/tests/qemu-iotests/tests/zoned.sh > > > @@ -0,0 +1,69 @@ > > > +#!/usr/bin/env bash > > > +# > > > +# Test zone management operations. > > > +# > > > + > > > +seq="$(basename $0)" > > > +echo "QA output created by $seq" > > > +status=1 # failure is the default! > > > + > > > +_cleanup() > > > +{ > > > + _cleanup_test_img > > > + sudo rmmod null_blk > > > +} > > > +trap "_cleanup; exit \$status" 0 1 2 3 15 > > > + > > > +# get standard environment, filters and checks > > > +. ./common.rc > > > +. ./common.filter > > > +. ./common.qemu > > > + > > > +# This test only runs on Linux hosts with raw image files. > > > +_supported_fmt raw > > > +_supported_proto file > > > +_supported_os Linux > > > + > > > +QEMU_IO="build/qemu-io" > > > +IMG="--image-opts driver=zoned_host_device,filename=/dev/nullb0" > > > +QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT > > > + > > > +echo "Testing a null_blk device" > > > +echo "Simple cases: if the operations work" > > > +sudo modprobe null_blk nr_devices=1 zoned=1 > > > > Jens & Ming Lei: > > > > null_blk is not ideal for automated test cases because it requires > > root and is global. If two tests are run at the same time they will > > interfere with each other. There is no way to have independent > > instances of the null_blk kernel module and the /dev/nullb0 device on > > a single Linux system. I think this test case can be merged for now > > but if there's time I suggest investigating alternatives. > > > > Maybe the new Linux ublk_drv driver is a better choice if it supports > > unprivileged operation with multiple independent block devices (plus > > This can be one big topic, and IMO, the io path needs to be reviewed > wrt. security risk. Also if one block device is stuck, it shouldn't > affect others, so far it can be one problem at least in sync/writeback, > if one device is hang, writeback on all block device may not move on. > > > zoned storage!). It would be necessary to write a userspace null block > > device that supports zoned storage if that doesn't exist already. I > > have CCed Ming Lei and Jens Axboe for thoughts. > > IMO, it shouldn't be hard to simulate zoned from userspace, the > patches[1] for setting device parameters are just sent out, then zoned > parameter could be sent to driver with the added commands easily. > > Even ublk can be used to implement zoned target, such as, ublk is > exposed as one normal disk, but the backing disk could be one real > zoned/zns device. > > [1] https://lore.kernel.org/linux-block/20220727141628.985429-1-ming.lei@redhat.com/T/#mb5518cf418b63fb6ca649f0df199137e50907a29 Thanks for replying! For QEMU testing purposes a null block driver with zone emulation is ideal. It does not need to persist data or zone metadata. It shouldn't require a backing device so that testing can be performed even without real zoned storage devices. It should also be possible to extend Linux null_blk to make it more friendly for parallel tests that are run without knowledge of each other/cooperation. But I was thinking ublk may have that infrastructure already. Stefan
diff --git a/tests/qemu-iotests/tests/zoned.sh b/tests/qemu-iotests/tests/zoned.sh new file mode 100755 index 0000000000..e14a3a420e --- /dev/null +++ b/tests/qemu-iotests/tests/zoned.sh @@ -0,0 +1,69 @@ +#!/usr/bin/env bash +# +# Test zone management operations. +# + +seq="$(basename $0)" +echo "QA output created by $seq" +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img + sudo rmmod null_blk +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter +. ./common.qemu + +# This test only runs on Linux hosts with raw image files. +_supported_fmt raw +_supported_proto file +_supported_os Linux + +QEMU_IO="build/qemu-io" +IMG="--image-opts driver=zoned_host_device,filename=/dev/nullb0" +QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT + +echo "Testing a null_blk device" +echo "Simple cases: if the operations work" +sudo modprobe null_blk nr_devices=1 zoned=1 +# hidden issues: +# 1. memory allocation error of "unaligned tcache chunk detected" when the nr_zone=1 in zone report +# 2. qemu-io: after report 10 zones, the program failed at double free error and exited. +echo "report the first zone" +sudo $QEMU_IO $IMG -c "zr 0 0 1" +echo "report: the first 10 zones" +sudo $QEMU_IO $IMG -c "zr 0 0 10" + +echo "open the first zone" +sudo $QEMU_IO $IMG -c "zo 0 0x80000" +echo "report after:" +sudo $QEMU_IO $IMG -c "zr 0 0 1" +echo "open the last zone" +sudo $QEMU_IO $IMG -c "zo 0x3e70000000 0x80000" +echo "report after:" +sudo $QEMU_IO $IMG -c "zr 0x3e70000000 0 2" + +echo "close the first zone" +sudo $QEMU_IO $IMG -c "zc 0 0x80000" +echo "report after:" +sudo $QEMU_IO $IMG -c "zr 0 0 1" +echo "close the last zone" +sudo $QEMU_IO $IMG -c "zc 0x3e70000000 0x80000" +echo "report after:" +sudo $QEMU_IO $IMG -c "zr 0x3e70000000 0 2" + + +echo "reset the second zone" +sudo $QEMU_IO $IMG -c "zrs 0x80000 0x80000" +echo "After resetting a zone:" +sudo $QEMU_IO $IMG -c "zr 0x80000 0 5" + +# success, all done +echo "*** done" +rm -f $seq.full +status=0
We have added new block layer APIs of zoned block devices. Test it with: (1) Create a null_blk device, run each zone operation on it and see whether reporting right zone information. Signed-off-by: Sam Li <faithilikerun@gmail.com> --- tests/qemu-iotests/tests/zoned.sh | 69 +++++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) create mode 100755 tests/qemu-iotests/tests/zoned.sh