Message ID | 20211227061114.54326-1-wqu@suse.com |
---|---|
State | Accepted |
Commit | 7c075433feb95b34e762edd384e76a5d08f73ad9 |
Delegated to: | Tom Rini |
Headers | show |
Series | fs/btrfs: fix a bug that U-boot fs btrfs implementation doesn't handle NO_HOLE feature correctly | expand |
On 2021/12/27 14:11, Qu Wenruo wrote: > [BUG] > When passing a btrfs with NO_HOLE feature to U-boot, and if one file > contains holes, then the hash of the file is not correct in U-boot: > > # mkfs.btrfs -f test.img # Since v5.15, mkfs defaults to NO_HOLES > # mount test.img /mnt/btrfs > # xfs_io -f -c "pwrite 0 4k" -c "pwrite 8k 4k" /mnt/btrfs/file > # md5sum /mnt/btrfs/file > 277f3840b275c74d01e979ea9d75ac19 /mnt/btrfs/file > # umount /mnt/btrfs > # ./u-boot > => host bind 0 /home/adam/test.img > => ls host 0 > < > 12288 Mon Dec 27 05:35:23 2021 file > => load host 0 0x1000000 file > 12288 bytes read in 0 ms > => md5sum 0x1000000 0x3000 > md5 for 01000000 ... 01002fff ==> 855ffdbe4d0ccc5acab92e1b5330e4c1 > > The md5sum doesn't match at all. > > [CAUSE] > In U-boot btrfs implementation, the function btrfs_read_file() has the > following iteration for file extent iteration: > > /* Read the aligned part */ > while (cur < aligned_end) { > ret = lookup_data_extent(root, &path, ino, cur, &next_offset); > if (ret < 0) > goto out; > if (ret > 0) { > /* No next, direct exit */ > if (!next_offset) { > ret = 0; > goto out; > } > } > /* Read file extent */ > > But for NO_HOLES features, hole extents will not have any extent item > for it. > Thus if @cur is at a hole, lookup_data_extent() will just return >0, and > update @next_offset. > > But we still believe there is some data to read for @cur for ret > 0 > case, causing we read extent data from the next file extent. > > This means, what we do for above NO_HOLES btrfs is: > - Read 4K data from disk to file offset [0, 4K) > So far the data is still correct > > - Read 4K data from disk to file offset [4K, 8K) > We didn't skip the 4K hole, but read the data at file offset [8K, 12K) > into file offset [4K, 8K). > > This causes the checksum mismatch. > > [FIX] > Add extra check to skip to the next non-hole range after > lookup_data_extent(). > > Signed-off-by: Qu Wenruo <wqu@suse.com> > --- > This bug exposed another missing link, that we don't have good test > coverage in U-boot btrfs. > > This is partially caused by the fact that, btrfs-progs code is not > designed to read file contents, but just to check the cross-reference > (aka, btrfs-check). > > If we really only want read-only support in U-boot, and don't ever plan > to add write support, then I'd say the btrfs-fuse project > (https://github.com/adam900710/btrfs-fuse/) is more suitable for U-boot. > > As that project already has full fs content verification selftest along > with extra multi-device recovery tests. > And shares the same code style between btrfs-progs/kernel. OK, things are not that bad. In fact, the btrfs_read_file() implementation in btrfs-fuse has the same naming, same lookup_file_extent() (just a little naming different than lookup_data_extent()), same parameter list. Just without the unaligned sector handling (handled by FUSE, and it may also be unnecessary for U-boot too), and already have the correct handling for lookup_file_extent(), thanks to the selftest. So this already means, it can be pretty easy for U-boot to take code from btrfs-fuse part by part, without huge refactor again. Thanks, Qu > --- > fs/btrfs/inode.c | 8 ++++++++ > 1 file changed, 8 insertions(+) > > diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c > index 2c2379303d74..d00b5153336d 100644 > --- a/fs/btrfs/inode.c > +++ b/fs/btrfs/inode.c > @@ -717,6 +717,14 @@ int btrfs_file_read(struct btrfs_root *root, u64 ino, u64 file_offset, u64 len, > ret = 0; > goto out; > } > + /* > + * Find a extent gap, mostly caused by NO_HOLE feature. > + * Just to next offset directly. > + */ > + if (next_offset > cur) { > + cur = next_offset; > + continue; > + } > } > fi = btrfs_item_ptr(path.nodes[0], path.slots[0], > struct btrfs_file_extent_item);
On Mon, Dec 27, 2021 at 02:11:14PM +0800, Qu Wenruo wrote: > [BUG] > When passing a btrfs with NO_HOLE feature to U-boot, and if one file > contains holes, then the hash of the file is not correct in U-boot: > > # mkfs.btrfs -f test.img # Since v5.15, mkfs defaults to NO_HOLES > # mount test.img /mnt/btrfs > # xfs_io -f -c "pwrite 0 4k" -c "pwrite 8k 4k" /mnt/btrfs/file > # md5sum /mnt/btrfs/file > 277f3840b275c74d01e979ea9d75ac19 /mnt/btrfs/file > # umount /mnt/btrfs > # ./u-boot > => host bind 0 /home/adam/test.img > => ls host 0 > < > 12288 Mon Dec 27 05:35:23 2021 file > => load host 0 0x1000000 file > 12288 bytes read in 0 ms > => md5sum 0x1000000 0x3000 > md5 for 01000000 ... 01002fff ==> 855ffdbe4d0ccc5acab92e1b5330e4c1 > > The md5sum doesn't match at all. > > [CAUSE] > In U-boot btrfs implementation, the function btrfs_read_file() has the > following iteration for file extent iteration: > > /* Read the aligned part */ > while (cur < aligned_end) { > ret = lookup_data_extent(root, &path, ino, cur, &next_offset); > if (ret < 0) > goto out; > if (ret > 0) { > /* No next, direct exit */ > if (!next_offset) { > ret = 0; > goto out; > } > } > /* Read file extent */ > > But for NO_HOLES features, hole extents will not have any extent item > for it. > Thus if @cur is at a hole, lookup_data_extent() will just return >0, and > update @next_offset. > > But we still believe there is some data to read for @cur for ret > 0 > case, causing we read extent data from the next file extent. > > This means, what we do for above NO_HOLES btrfs is: > - Read 4K data from disk to file offset [0, 4K) > So far the data is still correct > > - Read 4K data from disk to file offset [4K, 8K) > We didn't skip the 4K hole, but read the data at file offset [8K, 12K) > into file offset [4K, 8K). > > This causes the checksum mismatch. > > [FIX] > Add extra check to skip to the next non-hole range after > lookup_data_extent(). > > Signed-off-by: Qu Wenruo <wqu@suse.com> Applied to u-boot/master, thanks!
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 2c2379303d74..d00b5153336d 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -717,6 +717,14 @@ int btrfs_file_read(struct btrfs_root *root, u64 ino, u64 file_offset, u64 len, ret = 0; goto out; } + /* + * Find a extent gap, mostly caused by NO_HOLE feature. + * Just to next offset directly. + */ + if (next_offset > cur) { + cur = next_offset; + continue; + } } fi = btrfs_item_ptr(path.nodes[0], path.slots[0], struct btrfs_file_extent_item);
[BUG] When passing a btrfs with NO_HOLE feature to U-boot, and if one file contains holes, then the hash of the file is not correct in U-boot: # mkfs.btrfs -f test.img # Since v5.15, mkfs defaults to NO_HOLES # mount test.img /mnt/btrfs # xfs_io -f -c "pwrite 0 4k" -c "pwrite 8k 4k" /mnt/btrfs/file # md5sum /mnt/btrfs/file 277f3840b275c74d01e979ea9d75ac19 /mnt/btrfs/file # umount /mnt/btrfs # ./u-boot => host bind 0 /home/adam/test.img => ls host 0 < > 12288 Mon Dec 27 05:35:23 2021 file => load host 0 0x1000000 file 12288 bytes read in 0 ms => md5sum 0x1000000 0x3000 md5 for 01000000 ... 01002fff ==> 855ffdbe4d0ccc5acab92e1b5330e4c1 The md5sum doesn't match at all. [CAUSE] In U-boot btrfs implementation, the function btrfs_read_file() has the following iteration for file extent iteration: /* Read the aligned part */ while (cur < aligned_end) { ret = lookup_data_extent(root, &path, ino, cur, &next_offset); if (ret < 0) goto out; if (ret > 0) { /* No next, direct exit */ if (!next_offset) { ret = 0; goto out; } } /* Read file extent */ But for NO_HOLES features, hole extents will not have any extent item for it. Thus if @cur is at a hole, lookup_data_extent() will just return >0, and update @next_offset. But we still believe there is some data to read for @cur for ret > 0 case, causing we read extent data from the next file extent. This means, what we do for above NO_HOLES btrfs is: - Read 4K data from disk to file offset [0, 4K) So far the data is still correct - Read 4K data from disk to file offset [4K, 8K) We didn't skip the 4K hole, but read the data at file offset [8K, 12K) into file offset [4K, 8K). This causes the checksum mismatch. [FIX] Add extra check to skip to the next non-hole range after lookup_data_extent(). Signed-off-by: Qu Wenruo <wqu@suse.com> --- This bug exposed another missing link, that we don't have good test coverage in U-boot btrfs. This is partially caused by the fact that, btrfs-progs code is not designed to read file contents, but just to check the cross-reference (aka, btrfs-check). If we really only want read-only support in U-boot, and don't ever plan to add write support, then I'd say the btrfs-fuse project (https://github.com/adam900710/btrfs-fuse/) is more suitable for U-boot. As that project already has full fs content verification selftest along with extra multi-device recovery tests. And shares the same code style between btrfs-progs/kernel. --- fs/btrfs/inode.c | 8 ++++++++ 1 file changed, 8 insertions(+)