mbox series

[0/6] ext4: rework bigalloc reserved cluster accounting

Message ID 20180907204529.1662-1-enwlinux@gmail.com
Headers show
Series ext4: rework bigalloc reserved cluster accounting | expand

Message

Eric Whitney Sept. 7, 2018, 8:45 p.m. UTC
Ext4 can make gross overestimates of the number of reserved clusters
required to handle bigalloc write requests under delayed allocation,
resulting in premature ENOSPC and quota limit failures.

These patches fix the implementation of reserved cluster accounting for
bigalloc file systems configured to use delayed allocation, and fix the
specific bugs described in bugzilla #151491.

These patches also supersede a previous RFC posting from May 2018
("ext4: rework delayed allocated cluster accounting"), as well as a more
recent RFC posting from August 2018 with the same title as this series.

Eric Whitney (6):
  ext4: generalize extents status tree search functions
  ext4: add new pending reservation mechanism
  ext4: fix reserved cluster accounting at delayed write time
  ext4: reduce reserved cluster count by number of allocated clusters
  ext4: adjust reserved cluster count when removing extents
  ext4: fix reserved cluster accounting at page invalidation time

 fs/ext4/ext4.h              |  10 +-
 fs/ext4/ext4_extents.h      |  13 +
 fs/ext4/extents.c           | 602 ++++++++++++++++++++--------------------
 fs/ext4/extents_status.c    | 662 ++++++++++++++++++++++++++++++++++++++++++--
 fs/ext4/extents_status.h    |  84 +++++-
 fs/ext4/inode.c             | 113 +++++---
 fs/ext4/mballoc.c           |  14 +-
 fs/ext4/super.c             |   8 +
 include/trace/events/ext4.h |  99 +++++--
 9 files changed, 1207 insertions(+), 398 deletions(-)

Comments

Eric Whitney Sept. 11, 2018, 7 p.m. UTC | #1
* Eric Whitney <enwlinux@gmail.com>:
> Ext4 can make gross overestimates of the number of reserved clusters
> required to handle bigalloc write requests under delayed allocation,
> resulting in premature ENOSPC and quota limit failures.
> 
> These patches fix the implementation of reserved cluster accounting for
> bigalloc file systems configured to use delayed allocation, and fix the
> specific bugs described in bugzilla #151491.
> 
> These patches also supersede a previous RFC posting from May 2018
> ("ext4: rework delayed allocated cluster accounting"), as well as a more
> recent RFC posting from August 2018 with the same title as this series.
> 
> Eric Whitney (6):
>   ext4: generalize extents status tree search functions
>   ext4: add new pending reservation mechanism
>   ext4: fix reserved cluster accounting at delayed write time
>   ext4: reduce reserved cluster count by number of allocated clusters
>   ext4: adjust reserved cluster count when removing extents
>   ext4: fix reserved cluster accounting at page invalidation time
> 
>  fs/ext4/ext4.h              |  10 +-
>  fs/ext4/ext4_extents.h      |  13 +
>  fs/ext4/extents.c           | 602 ++++++++++++++++++++--------------------
>  fs/ext4/extents_status.c    | 662 ++++++++++++++++++++++++++++++++++++++++++--
>  fs/ext4/extents_status.h    |  84 +++++-
>  fs/ext4/inode.c             | 113 +++++---
>  fs/ext4/mballoc.c           |  14 +-
>  fs/ext4/super.c             |   8 +
>  include/trace/events/ext4.h |  99 +++++--
>  9 files changed, 1207 insertions(+), 398 deletions(-)
> 
> -- 
> 2.11.0
>

An FYI for reviewers -

I've got a v2 for this series in testing.  It fixes some bugs that lead to
infrequent test failures for generic/112 when running the kvm-xfstests
test appliance on the bigalloc and bigalloc_1k test cases.  Basically,
fallocated extents weren't always being handled correctly.  Also,  I've
corrected some misleading nomenclature and a few comments.  These changes
are small and limited to patches 2, 4, and 6 - review effort invested in
v1 should be largely applicable to v2.  The patches look very solid at this
point.

I expect to post v2 tomorrow once a complete test appliance run has
finished.

Thanks,
Eric