mbox

[PULL,00/17] Block patches

Message ID 20230428123954.179035-1-stefanha@redhat.com
State New
Headers show

Pull-request

https://gitlab.com/stefanha/qemu.git tags/block-pull-request

Message

Stefan Hajnoczi April 28, 2023, 12:39 p.m. UTC 
The following changes since commit 05d50ba2d4668d43a835c5a502efdec9b92646e6:

  Merge tag 'migration-20230427-pull-request' of https://gitlab.com/juan.quintela/qemu into staging (2023-04-28 08:35:06 +0100)

are available in the Git repository at:

  https://gitlab.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to d3c760be786571d83d5cea01953e543df4d76f51:

  docs/zoned-storage:add zoned emulation use case (2023-04-28 08:34:07 -0400)

----------------------------------------------------------------
Pull request

This pull request contains Sam Li's virtio-blk zoned storage work. These
patches were dropped from my previous block pull request due to CI failures.

----------------------------------------------------------------

Sam Li (17):
  block/block-common: add zoned device structs
  block/file-posix: introduce helper functions for sysfs attributes
  block/block-backend: add block layer APIs resembling Linux
    ZonedBlockDevice ioctls
  block/raw-format: add zone operations to pass through requests
  block: add zoned BlockDriver check to block layer
  iotests: test new zone operations
  block: add some trace events for new block layer APIs
  docs/zoned-storage: add zoned device documentation
  file-posix: add tracking of the zone write pointers
  block: introduce zone append write for zoned devices
  qemu-iotests: test zone append operation
  block: add some trace events for zone append
  include: update virtio_blk headers to v6.3-rc1
  virtio-blk: add zoned storage emulation for zoned devices
  block: add accounting for zone append operation
  virtio-blk: add some trace events for zoned emulation
  docs/zoned-storage:add zoned emulation use case

 docs/devel/index-api.rst                     |   1 +
 docs/devel/zoned-storage.rst                 |  62 ++
 qapi/block-core.json                         |  68 +-
 qapi/block.json                              |   4 +
 meson.build                                  |   4 +
 include/block/accounting.h                   |   1 +
 include/block/block-common.h                 |  57 ++
 include/block/block-io.h                     |  13 +
 include/block/block_int-common.h             |  37 +
 include/block/raw-aio.h                      |   8 +-
 include/standard-headers/drm/drm_fourcc.h    |  12 +
 include/standard-headers/linux/ethtool.h     |  48 +-
 include/standard-headers/linux/fuse.h        |  45 +-
 include/standard-headers/linux/pci_regs.h    |   1 +
 include/standard-headers/linux/vhost_types.h |   2 +
 include/standard-headers/linux/virtio_blk.h  | 105 +++
 include/sysemu/block-backend-io.h            |  27 +
 linux-headers/asm-arm64/kvm.h                |   1 +
 linux-headers/asm-x86/kvm.h                  |  34 +-
 linux-headers/linux/kvm.h                    |   9 +
 linux-headers/linux/vfio.h                   |  15 +-
 linux-headers/linux/vhost.h                  |   8 +
 block.c                                      |  19 +
 block/block-backend.c                        | 198 ++++++
 block/file-posix.c                           | 696 +++++++++++++++++--
 block/io.c                                   |  68 ++
 block/io_uring.c                             |   4 +
 block/linux-aio.c                            |   3 +
 block/qapi-sysemu.c                          |  11 +
 block/qapi.c                                 |  18 +
 block/raw-format.c                           |  26 +
 hw/block/virtio-blk-common.c                 |   2 +
 hw/block/virtio-blk.c                        | 405 +++++++++++
 hw/virtio/virtio-qmp.c                       |   2 +
 qemu-io-cmds.c                               | 224 ++++++
 block/trace-events                           |   4 +
 docs/system/qemu-block-drivers.rst.inc       |   6 +
 hw/block/trace-events                        |   7 +
 tests/qemu-iotests/tests/zoned               | 105 +++
 tests/qemu-iotests/tests/zoned.out           |  69 ++
 40 files changed, 2361 insertions(+), 68 deletions(-)
 create mode 100644 docs/devel/zoned-storage.rst
 create mode 100755 tests/qemu-iotests/tests/zoned
 create mode 100644 tests/qemu-iotests/tests/zoned.out

Comments

Richard Henderson April 29, 2023, 10:05 p.m. UTC | #1 
On 4/28/23 13:39, Stefan Hajnoczi wrote:
> The following changes since commit 05d50ba2d4668d43a835c5a502efdec9b92646e6:
> 
>    Merge tag 'migration-20230427-pull-request' of https://gitlab.com/juan.quintela/qemu into staging (2023-04-28 08:35:06 +0100)
> 
> are available in the Git repository at:
> 
>    https://gitlab.com/stefanha/qemu.git tags/block-pull-request
> 
> for you to fetch changes up to d3c760be786571d83d5cea01953e543df4d76f51:
> 
>    docs/zoned-storage:add zoned emulation use case (2023-04-28 08:34:07 -0400)
> 
> ----------------------------------------------------------------
> Pull request
> 
> This pull request contains Sam Li's virtio-blk zoned storage work. These
> patches were dropped from my previous block pull request due to CI failures.


More CI build failures, e.g.

https://gitlab.com/qemu-project/qemu/-/jobs/4202086013#L1720
https://gitlab.com/qemu-project/qemu/-/jobs/4202085995#L4088


r~
Stefan Hajnoczi May 1, 2023, 11:56 a.m. UTC | #2 
On Sat, Apr 29, 2023 at 11:05:06PM +0100, Richard Henderson wrote:
> On 4/28/23 13:39, Stefan Hajnoczi wrote:
> > The following changes since commit 05d50ba2d4668d43a835c5a502efdec9b92646e6:
> > 
> >    Merge tag 'migration-20230427-pull-request' of https://gitlab.com/juan.quintela/qemu into staging (2023-04-28 08:35:06 +0100)
> > 
> > are available in the Git repository at:
> > 
> >    https://gitlab.com/stefanha/qemu.git tags/block-pull-request
> > 
> > for you to fetch changes up to d3c760be786571d83d5cea01953e543df4d76f51:
> > 
> >    docs/zoned-storage:add zoned emulation use case (2023-04-28 08:34:07 -0400)
> > 
> > ----------------------------------------------------------------
> > Pull request
> > 
> > This pull request contains Sam Li's virtio-blk zoned storage work. These
> > patches were dropped from my previous block pull request due to CI failures.
> 
> 
> More CI build failures, e.g.

Hi Sam,
There are some more CI failures.

> 
> https://gitlab.com/qemu-project/qemu/-/jobs/4202086013#L1720

This Ubuntu 20.04 on s390x CI job failed because <linux/blkzoned.h> is
missing Linux commit e876df1fe0ad ("block: add zone open, close and
finish ioctl support"):

  ../block/file-posix.c: In function ‘raw_co_zone_mgmt’:
  ../block/file-posix.c:3472:14: error: ‘BLKOPENZONE’ undeclared (first use in this function)
   3472 |         zo = BLKOPENZONE;
        |              ^~~~~~~~~~~
  ../block/file-posix.c:3472:14: note: each undeclared identifier is reported only once for each function it appears in
  ../block/file-posix.c:3476:14: error: ‘BLKCLOSEZONE’ undeclared (first use in this function); did you mean ‘BLKRESETZONE’?
   3476 |         zo = BLKCLOSEZONE;
        |              ^~~~~~~~~~~~
        |              BLKRESETZONE
  ../block/file-posix.c:3480:14: error: ‘BLKFINISHZONE’ undeclared (first use in this function)
   3480 |         zo = BLKFINISHZONE;
        |              ^~~~~~~~~~~~~

Older kernels didn't have these ioctls. I don't think it makes sense to
enable file-posix zoned functionality without these ioctls.

I suggest changing the CONFIG_BLKZONED check in meson.build from:

  config_host_data.set('CONFIG_BLKZONED', cc.has_header('linux/blkzoned.h'))

 to:

  config_host_data.set('CONFIG_BLKZONED', cc.has_header_symbol('linux/blkzoned.h', 'BLKOPENZONE'))

> https://gitlab.com/qemu-project/qemu/-/jobs/4202085995#L4088

The <linux/asm/kvm.h> header file started using __DECLARE_FLEX_ARRAY()
and QEMU doesn't have that macro:

  linux-headers/asm/kvm.h:509:3: error: expected specifier-qualifier-list before '__DECLARE_FLEX_ARRAY'
    509 |   __DECLARE_FLEX_ARRAY(struct kvm_vmx_nested_state_data, vmx);
        |   ^~~~~~~~~~~~~~~~~~~~

You could update the sed command in scripts/update-linux-headers.sh to
convert __DECLARE_FLEX_ARRAY(type, field) into type field[] or import
the Linux macro definition of __DECLARE_FLEX_ARRAY().

Another failure is
https://gitlab.com/qemu-project/qemu/-/jobs/4202085991 where
qemu-iotests is failing because the output has changed due to the
addition of zoned fields to block stats.

Another failure is
https://gitlab.com/qemu-project/qemu/-/jobs/4202086041 where
qemu-system-ppc seems to segfault.

You can find the CI results here: https://gitlab.com/qemu-project/qemu/-/pipelines/852908752

You can run the GitLab CI yourself like this:

  $ git push -o ci.variable=QEMU_CI=2 your_gitlab_repo HEAD

Stefan