mbox series

[0/5] fs: ext4: implement opendir, readdir, closedir

Message ID 20241026064048.370062-1-heinrich.schuchardt@canonical.com
Headers show
Series fs: ext4: implement opendir, readdir, closedir | expand

Message

Heinrich Schuchardt Oct. 26, 2024, 6:40 a.m. UTC
With this series opendir, readdir, closedir are implemented for ext4.
These functions are needed for the UEFI sub-system to interact with
the ext4 file system.

To reduce code growth the functions are reused to implement the ls
command for ext4.

A memory leak in ext4fs_exists is resolved.

ext4fs_iterate_dir is simplified by removing a redundant pointer copy.

Heinrich Schuchardt (5):
  fs: ext4: simplify ext4fs_iterate_dir()
  fs: ext4: free directory node in ext4fs_exists()
  fs: ext4: implement opendir, readdir, closedir
  efi_loader: fix GetInfo and SetInfo
  fs: ext4: use fs_ls_generic

 fs/ext4/ext4_common.c               |  48 ++------
 fs/ext4/ext4fs.c                    | 177 +++++++++++++++++++++++++---
 fs/fs.c                             |   6 +-
 include/ext4fs.h                    |   4 +
 lib/efi_loader/efi_file.c           |  30 +++--
 test/py/tests/test_env.py           |   2 +-
 test/py/tests/test_fs/test_basic.py |   5 +-
 7 files changed, 197 insertions(+), 75 deletions(-)

Comments

Tom Rini Nov. 2, 2024, 11:25 p.m. UTC | #1
On Sat, 26 Oct 2024 08:40:43 +0200, Heinrich Schuchardt wrote:

> With this series opendir, readdir, closedir are implemented for ext4.
> These functions are needed for the UEFI sub-system to interact with
> the ext4 file system.
> 
> To reduce code growth the functions are reused to implement the ls
> command for ext4.
> 
> [...]

Applied to local tree/v2-tidy-test-dir, thanks!
Michael Nazzareno Trimarchi Nov. 2, 2024, 11:36 p.m. UTC | #2
Hi Tom

On Sun, Nov 3, 2024 at 12:26 AM Tom Rini <trini@konsulko.com> wrote:
>
> On Sat, 26 Oct 2024 08:40:43 +0200, Heinrich Schuchardt wrote:
>
> > With this series opendir, readdir, closedir are implemented for ext4.
> > These functions are needed for the UEFI sub-system to interact with
> > the ext4 file system.
> >
> > To reduce code growth the functions are reused to implement the ls
> > command for ext4.
> >
> > [...]
>
> Applied to local tree/v2-tidy-test-dir, thanks!
>
Am I sleeping?

int ext4fs_opendir(const char *dirname, struct fs_dir_stream **dirsp) {
    struct ext4_dir_stream *dirs = NULL;
    struct ext2fs_node *dir = NULL;
    int ret;

    *dirsp = NULL;

    dirs = calloc(1, sizeof(struct ext4_dir_stream));
    if (!dirs)
        return -ENOMEM;

    dirs->dirname = strdup(dirname);
    if (!dirs->dirname) {
        free(dirs);
        return -ENOMEM;
    }

    ret = ext4fs_find_file(dirname, &ext4fs_root->diropen, &dir,
FILETYPE_DIRECTORY);

    if (ret == 1) {
        ret = 0;
        *dirsp = (struct fs_dir_stream *)dirs;
    } else {
        ret = -ENOENT;
        free(dirs->dirname);
        free(dirs);
    }

    if (dir)
        ext4fs_free_node(dir, &ext4fs_root->diropen);

    return ret;
}

Should not be like this?

> --
> Tom
>
>
Tom Rini Nov. 2, 2024, 11:53 p.m. UTC | #3
On Sun, Nov 03, 2024 at 12:36:38AM +0100, Michael Nazzareno Trimarchi wrote:
> Hi Tom
> 
> On Sun, Nov 3, 2024 at 12:26 AM Tom Rini <trini@konsulko.com> wrote:
> >
> > On Sat, 26 Oct 2024 08:40:43 +0200, Heinrich Schuchardt wrote:
> >
> > > With this series opendir, readdir, closedir are implemented for ext4.
> > > These functions are needed for the UEFI sub-system to interact with
> > > the ext4 file system.
> > >
> > > To reduce code growth the functions are reused to implement the ls
> > > command for ext4.
> > >
> > > [...]
> >
> > Applied to local tree/v2-tidy-test-dir, thanks!
> >
> Am I sleeping?

... I always forget b4 compares with checked out branch, that should
have been master.

> int ext4fs_opendir(const char *dirname, struct fs_dir_stream **dirsp) {
>     struct ext4_dir_stream *dirs = NULL;
>     struct ext2fs_node *dir = NULL;
>     int ret;
> 
>     *dirsp = NULL;
> 
>     dirs = calloc(1, sizeof(struct ext4_dir_stream));
>     if (!dirs)
>         return -ENOMEM;
> 
>     dirs->dirname = strdup(dirname);
>     if (!dirs->dirname) {
>         free(dirs);
>         return -ENOMEM;
>     }
> 
>     ret = ext4fs_find_file(dirname, &ext4fs_root->diropen, &dir,
> FILETYPE_DIRECTORY);
> 
>     if (ret == 1) {
>         ret = 0;
>         *dirsp = (struct fs_dir_stream *)dirs;
>     } else {
>         ret = -ENOENT;
>         free(dirs->dirname);
>         free(dirs);
>     }
> 
>     if (dir)
>         ext4fs_free_node(dir, &ext4fs_root->diropen);
> 
>     return ret;
> }
> 
> Should not be like this?

Please elaborate?
Michael Nazzareno Trimarchi Nov. 3, 2024, 8:21 a.m. UTC | #4
Hi

Il dom 3 nov 2024, 00:53 Tom Rini <trini@konsulko.com> ha scritto:

> On Sun, Nov 03, 2024 at 12:36:38AM +0100, Michael Nazzareno Trimarchi
> wrote:
> > Hi Tom
> >
> > On Sun, Nov 3, 2024 at 12:26 AM Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Sat, 26 Oct 2024 08:40:43 +0200, Heinrich Schuchardt wrote:
> > >
> > > > With this series opendir, readdir, closedir are implemented for ext4.
> > > > These functions are needed for the UEFI sub-system to interact with
> > > > the ext4 file system.
> > > >
> > > > To reduce code growth the functions are reused to implement the ls
> > > > command for ext4.
> > > >
> > > > [...]
> > >
> > > Applied to local tree/v2-tidy-test-dir, thanks!
> > >
> > Am I sleeping?
>
> ... I always forget b4 compares with checked out branch, that should
> have been master.
>
> > int ext4fs_opendir(const char *dirname, struct fs_dir_stream **dirsp) {
> >     struct ext4_dir_stream *dirs = NULL;
> >     struct ext2fs_node *dir = NULL;
> >     int ret;
> >
> >     *dirsp = NULL;
> >
> >     dirs = calloc(1, sizeof(struct ext4_dir_stream));
> >     if (!dirs)
> >         return -ENOMEM;
> >
> >     dirs->dirname = strdup(dirname);
> >     if (!dirs->dirname) {
> >         free(dirs);
> >         return -ENOMEM;
> >     }
> >
> >     ret = ext4fs_find_file(dirname, &ext4fs_root->diropen, &dir,
> > FILETYPE_DIRECTORY);
> >
> >     if (ret == 1) {
> >         ret = 0;
> >         *dirsp = (struct fs_dir_stream *)dirs;
> >     } else {
> >         ret = -ENOENT;
> >         free(dirs->dirname);
> >         free(dirs);
>

I have add in this path the free of dirs, because according to the code I
can see from this email is not reference anymore and anyway according to
what I have commented already moving allocation of dirs later make the code
a bit simpler.

Michael


>     }
> >
> >     if (dir)
> >         ext4fs_free_node(dir, &ext4fs_root->diropen);
> >
> >     return ret;
> > }
> >
> > Should not be like this?
>
> Please elaborate?
>
> --
> Tom
>
Michael Nazzareno Trimarchi Nov. 7, 2024, 9:11 a.m. UTC | #5
Hi Tom

On Sun, Nov 3, 2024 at 9:21 AM Michael Nazzareno Trimarchi
<michael@amarulasolutions.com> wrote:
>
> Hi
>
> Il dom 3 nov 2024, 00:53 Tom Rini <trini@konsulko.com> ha scritto:
>>
>> On Sun, Nov 03, 2024 at 12:36:38AM +0100, Michael Nazzareno Trimarchi wrote:
>> > Hi Tom
>> >
>> > On Sun, Nov 3, 2024 at 12:26 AM Tom Rini <trini@konsulko.com> wrote:
>> > >
>> > > On Sat, 26 Oct 2024 08:40:43 +0200, Heinrich Schuchardt wrote:
>> > >
>> > > > With this series opendir, readdir, closedir are implemented for ext4.
>> > > > These functions are needed for the UEFI sub-system to interact with
>> > > > the ext4 file system.
>> > > >
>> > > > To reduce code growth the functions are reused to implement the ls
>> > > > command for ext4.
>> > > >
>> > > > [...]
>> > >
>> > > Applied to local tree/v2-tidy-test-dir, thanks!
>> > >
>> > Am I sleeping?
>>
>> ... I always forget b4 compares with checked out branch, that should
>> have been master.
>>
>> > int ext4fs_opendir(const char *dirname, struct fs_dir_stream **dirsp) {
>> >     struct ext4_dir_stream *dirs = NULL;
>> >     struct ext2fs_node *dir = NULL;
>> >     int ret;
>> >
>> >     *dirsp = NULL;
>> >
>> >     dirs = calloc(1, sizeof(struct ext4_dir_stream));
>> >     if (!dirs)
>> >         return -ENOMEM;
>> >
>> >     dirs->dirname = strdup(dirname);
>> >     if (!dirs->dirname) {
>> >         free(dirs);
>> >         return -ENOMEM;
>> >     }
>> >
>> >     ret = ext4fs_find_file(dirname, &ext4fs_root->diropen, &dir,
>> > FILETYPE_DIRECTORY);
>> >
>> >     if (ret == 1) {
>> >         ret = 0;
>> >         *dirsp = (struct fs_dir_stream *)dirs;
>> >     } else {
>> >         ret = -ENOENT;
>> >         free(dirs->dirname);
>> >         free(dirs);
>
>
> I have add in this path the free of dirs, because according to the code I can see from this email is not reference anymore and anyway according to what I have commented already moving allocation of dirs later make the code a bit simpler.
>

There are few leaks in this function. I have fixed, can you take a
look again on what you merge?

Michael

> Michael
>
>
>> >     }
>> >
>> >     if (dir)
>> >         ext4fs_free_node(dir, &ext4fs_root->diropen);
>> >
>> >     return ret;
>> > }
>> >
>> > Should not be like this?
>>
>> Please elaborate?
>>
>> --
>> Tom
Heinrich Schuchardt Nov. 7, 2024, 9:27 a.m. UTC | #6
On 11/3/24 00:36, Michael Nazzareno Trimarchi wrote:
> Hi Tom
>
> On Sun, Nov 3, 2024 at 12:26 AM Tom Rini <trini@konsulko.com> wrote:
>>
>> On Sat, 26 Oct 2024 08:40:43 +0200, Heinrich Schuchardt wrote:
>>
>>> With this series opendir, readdir, closedir are implemented for ext4.
>>> These functions are needed for the UEFI sub-system to interact with
>>> the ext4 file system.
>>>
>>> To reduce code growth the functions are reused to implement the ls
>>> command for ext4.
>>>
>>> [...]
>>
>> Applied to local tree/v2-tidy-test-dir, thanks!
>>
> Am I sleeping?
>
> int ext4fs_opendir(const char *dirname, struct fs_dir_stream **dirsp) {
>      struct ext4_dir_stream *dirs = NULL;

Thank you for your review.

We should not set a value that we do not use.

>      struct ext2fs_node *dir = NULL;
>      int ret;
>
>      *dirsp = NULL;
>
>      dirs = calloc(1, sizeof(struct ext4_dir_stream));
>      if (!dirs)
>          return -ENOMEM;
>
>      dirs->dirname = strdup(dirname);
>      if (!dirs->dirname) {

Yes this line in my code is incorrect.

>          free(dirs);
>          return -ENOMEM;
>      }
>
>      ret = ext4fs_find_file(dirname, &ext4fs_root->diropen, &dir,
> FILETYPE_DIRECTORY);
>
>      if (ret == 1) {
>          ret = 0;
>          *dirsp = (struct fs_dir_stream *)dirs;
>      } else {
>          ret = -ENOENT;
>          free(dirs->dirname);
>          free(dirs);

Yes, we should avoid a memory leak here.

I will send a patch.

Best regards

Heinrich

>      }
>
>      if (dir)
>          ext4fs_free_node(dir, &ext4fs_root->diropen);
>
>      return ret;
> }
>
> Should not be like this?
>
>> --
>> Tom
>>
>>
>
>
Michael Nazzareno Trimarchi Nov. 7, 2024, 9:31 a.m. UTC | #7
Hi

On Thu, Nov 7, 2024 at 10:27 AM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>
> On 11/3/24 00:36, Michael Nazzareno Trimarchi wrote:
> > Hi Tom
> >
> > On Sun, Nov 3, 2024 at 12:26 AM Tom Rini <trini@konsulko.com> wrote:
> >>
> >> On Sat, 26 Oct 2024 08:40:43 +0200, Heinrich Schuchardt wrote:
> >>
> >>> With this series opendir, readdir, closedir are implemented for ext4.
> >>> These functions are needed for the UEFI sub-system to interact with
> >>> the ext4 file system.
> >>>
> >>> To reduce code growth the functions are reused to implement the ls
> >>> command for ext4.
> >>>
> >>> [...]
> >>
> >> Applied to local tree/v2-tidy-test-dir, thanks!
> >>
> > Am I sleeping?
> >
> > int ext4fs_opendir(const char *dirname, struct fs_dir_stream **dirsp) {
> >      struct ext4_dir_stream *dirs = NULL;
>
> Thank you for your review.
>
> We should not set a value that we do not use.
>
> >      struct ext2fs_node *dir = NULL;
> >      int ret;
> >
> >      *dirsp = NULL;
> >
> >      dirs = calloc(1, sizeof(struct ext4_dir_stream));
> >      if (!dirs)
> >          return -ENOMEM;
> >
> >      dirs->dirname = strdup(dirname);
> >      if (!dirs->dirname) {
>
> Yes this line in my code is incorrect.
>

We need to use valgrind in unit test or we need some code analys to
automate this process.
We are always looking for memory leaks in uboot in exit path or we
need to have alloction funciton that
automatic free the memory when we go in error path

Michael

> >          free(dirs);
> >          return -ENOMEM;
> >      }
> >
> >      ret = ext4fs_find_file(dirname, &ext4fs_root->diropen, &dir,
> > FILETYPE_DIRECTORY);
> >
> >      if (ret == 1) {
> >          ret = 0;
> >          *dirsp = (struct fs_dir_stream *)dirs;
> >      } else {
> >          ret = -ENOENT;
> >          free(dirs->dirname);
> >          free(dirs);
>
> Yes, we should avoid a memory leak here.
>
> I will send a patch.
>
> Best regards
>
> Heinrich
>
> >      }
> >
> >      if (dir)
> >          ext4fs_free_node(dir, &ext4fs_root->diropen);
> >
> >      return ret;
> > }
> >
> > Should not be like this?
> >
> >> --
> >> Tom
> >>
> >>
> >
> >
>
Heinrich Schuchardt Nov. 7, 2024, 9:47 a.m. UTC | #8
On 11/7/24 10:31, Michael Nazzareno Trimarchi wrote:
> Hi
>
> On Thu, Nov 7, 2024 at 10:27 AM Heinrich Schuchardt <xypron.glpk@gmx.de> wrote:
>>
>> On 11/3/24 00:36, Michael Nazzareno Trimarchi wrote:
>>> Hi Tom
>>>
>>> On Sun, Nov 3, 2024 at 12:26 AM Tom Rini <trini@konsulko.com> wrote:
>>>>
>>>> On Sat, 26 Oct 2024 08:40:43 +0200, Heinrich Schuchardt wrote:
>>>>
>>>>> With this series opendir, readdir, closedir are implemented for ext4.
>>>>> These functions are needed for the UEFI sub-system to interact with
>>>>> the ext4 file system.
>>>>>
>>>>> To reduce code growth the functions are reused to implement the ls
>>>>> command for ext4.
>>>>>
>>>>> [...]
>>>>
>>>> Applied to local tree/v2-tidy-test-dir, thanks!
>>>>
>>> Am I sleeping?
>>>
>>> int ext4fs_opendir(const char *dirname, struct fs_dir_stream **dirsp) {
>>>       struct ext4_dir_stream *dirs = NULL;
>>
>> Thank you for your review.
>>
>> We should not set a value that we do not use.
>>
>>>       struct ext2fs_node *dir = NULL;
>>>       int ret;
>>>
>>>       *dirsp = NULL;
>>>
>>>       dirs = calloc(1, sizeof(struct ext4_dir_stream));
>>>       if (!dirs)
>>>           return -ENOMEM;
>>>
>>>       dirs->dirname = strdup(dirname);
>>>       if (!dirs->dirname) {
>>
>> Yes this line in my code is incorrect.
>>
>
> We need to use valgrind in unit test or we need some code analys to
> automate this process.
> We are always looking for memory leaks in uboot in exit path or we
> need to have alloction funciton that
> automatic free the memory when we go in error path

Coverity probably would have reported this. But I think we only run it
on tags.

This article describes how Valgrind could be used:

https://developers.redhat.com/articles/2022/03/23/use-valgrind-memcheck-custom-memory-manager

Both malloc and LMB would be candidates.

Best regards

Heinrich

>
> Michael
>
>>>           free(dirs);
>>>           return -ENOMEM;
>>>       }
>>>
>>>       ret = ext4fs_find_file(dirname, &ext4fs_root->diropen, &dir,
>>> FILETYPE_DIRECTORY);
>>>
>>>       if (ret == 1) {
>>>           ret = 0;
>>>           *dirsp = (struct fs_dir_stream *)dirs;
>>>       } else {
>>>           ret = -ENOENT;
>>>           free(dirs->dirname);
>>>           free(dirs);
>>
>> Yes, we should avoid a memory leak here.
>>
>> I will send a patch.
>>
>> Best regards
>>
>> Heinrich
>>
>>>       }
>>>
>>>       if (dir)
>>>           ext4fs_free_node(dir, &ext4fs_root->diropen);
>>>
>>>       return ret;
>>> }
>>>
>>> Should not be like this?
>>>
>>>> --
>>>> Tom
>>>>
>>>>
>>>
>>>
>>
>
>