[RFC,7/8] tests/pytest: Add a function for extracting files from an archive

Some Avocado-based tests use the "archive" module from avocado.utils
to extract files from an archive. To be able to use these tests
without Avocado, we have to provide our own function for extracting
files. Fortunately, there is already the tarfile module that will
provide us with this functionality, so let's just add a nice wrapper
function around that.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/pytest/qemu_pytest/utils.py | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
 create mode 100644 tests/pytest/qemu_pytest/utils.py

On Thu, Jul 11, 2024 at 01:55:45PM +0200, Thomas Huth wrote:
> Some Avocado-based tests use the "archive" module from avocado.utils
> to extract files from an archive. To be able to use these tests
> without Avocado, we have to provide our own function for extracting
> files. Fortunately, there is already the tarfile module that will
> provide us with this functionality, so let's just add a nice wrapper
> function around that.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  tests/pytest/qemu_pytest/utils.py | 21 +++++++++++++++++++++
>  1 file changed, 21 insertions(+)
>  create mode 100644 tests/pytest/qemu_pytest/utils.py
> 
> diff --git a/tests/pytest/qemu_pytest/utils.py b/tests/pytest/qemu_pytest/utils.py
> new file mode 100644
> index 0000000000..4eb5e5d5e5
> --- /dev/null
> +++ b/tests/pytest/qemu_pytest/utils.py
> @@ -0,0 +1,21 @@
> +# Utilities for python-based QEMU tests
> +#
> +# Copyright 2024 Red Hat, Inc.
> +#
> +# Authors:
> +#  Thomas Huth <thuth@redhat.com>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or
> +# later.  See the COPYING file in the top-level directory.
> +
> +import tarfile
> +
> +def archive_extract(archive, dest_dir, member=None):
> +    with tarfile.open(archive) as tf:
> +        if hasattr(tarfile, 'data_filter'):

Not convinced this is still needed. The python docs don't say anything
about 'data_filter' being introduced after 3.0, so can likely
assume it always exists.

> +            tf.extraction_filter = getattr(tarfile, 'data_filter',
> +                                           (lambda member, path: member))
> +        if member:
> +            tf.extract(member=member, path=dest_dir)
> +        else:
> +            tf.extractall(path=dest_dir)
> -- 
> 2.45.2
> 

With regards,
Daniel

On 12/07/2024 11.14, Daniel P. Berrangé wrote:
> On Thu, Jul 11, 2024 at 01:55:45PM +0200, Thomas Huth wrote:
>> Some Avocado-based tests use the "archive" module from avocado.utils
>> to extract files from an archive. To be able to use these tests
>> without Avocado, we have to provide our own function for extracting
>> files. Fortunately, there is already the tarfile module that will
>> provide us with this functionality, so let's just add a nice wrapper
>> function around that.
>>
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>> ---
>>   tests/pytest/qemu_pytest/utils.py | 21 +++++++++++++++++++++
>>   1 file changed, 21 insertions(+)
>>   create mode 100644 tests/pytest/qemu_pytest/utils.py
>>
>> diff --git a/tests/pytest/qemu_pytest/utils.py b/tests/pytest/qemu_pytest/utils.py
>> new file mode 100644
>> index 0000000000..4eb5e5d5e5
>> --- /dev/null
>> +++ b/tests/pytest/qemu_pytest/utils.py
>> @@ -0,0 +1,21 @@
>> +# Utilities for python-based QEMU tests
>> +#
>> +# Copyright 2024 Red Hat, Inc.
>> +#
>> +# Authors:
>> +#  Thomas Huth <thuth@redhat.com>
>> +#
>> +# This work is licensed under the terms of the GNU GPL, version 2 or
>> +# later.  See the COPYING file in the top-level directory.
>> +
>> +import tarfile
>> +
>> +def archive_extract(archive, dest_dir, member=None):
>> +    with tarfile.open(archive) as tf:
>> +        if hasattr(tarfile, 'data_filter'):
> 
> Not convinced this is still needed. The python docs don't say anything
> about 'data_filter' being introduced after 3.0, so can likely
> assume it always exists.

According to https://docs.python.org/3/library/tarfile.html :

"Extraction filters were added to Python 3.12, but may be backported to 
older versions as security updates. To check whether the feature is 
available, use e.g. hasattr(tarfile, 'data_filter') rather than checking the 
Python version."

And it seems to be missing in Python 3.7, indeed:

  https://docs.python.org/3.7/library/tarfile.html

So as long as we still support this old version, I think I've got to keep 
this check.

  Thomas

On Fri, Jul 12, 2024 at 01:52:03PM +0200, Thomas Huth wrote:
> On 12/07/2024 11.14, Daniel P. Berrangé wrote:
> > On Thu, Jul 11, 2024 at 01:55:45PM +0200, Thomas Huth wrote:
> > > Some Avocado-based tests use the "archive" module from avocado.utils
> > > to extract files from an archive. To be able to use these tests
> > > without Avocado, we have to provide our own function for extracting
> > > files. Fortunately, there is already the tarfile module that will
> > > provide us with this functionality, so let's just add a nice wrapper
> > > function around that.
> > > 
> > > Signed-off-by: Thomas Huth <thuth@redhat.com>
> > > ---
> > >   tests/pytest/qemu_pytest/utils.py | 21 +++++++++++++++++++++
> > >   1 file changed, 21 insertions(+)
> > >   create mode 100644 tests/pytest/qemu_pytest/utils.py
> > > 
> > > diff --git a/tests/pytest/qemu_pytest/utils.py b/tests/pytest/qemu_pytest/utils.py
> > > new file mode 100644
> > > index 0000000000..4eb5e5d5e5
> > > --- /dev/null
> > > +++ b/tests/pytest/qemu_pytest/utils.py
> > > @@ -0,0 +1,21 @@
> > > +# Utilities for python-based QEMU tests
> > > +#
> > > +# Copyright 2024 Red Hat, Inc.
> > > +#
> > > +# Authors:
> > > +#  Thomas Huth <thuth@redhat.com>
> > > +#
> > > +# This work is licensed under the terms of the GNU GPL, version 2 or
> > > +# later.  See the COPYING file in the top-level directory.
> > > +
> > > +import tarfile
> > > +
> > > +def archive_extract(archive, dest_dir, member=None):
> > > +    with tarfile.open(archive) as tf:
> > > +        if hasattr(tarfile, 'data_filter'):
> > 
> > Not convinced this is still needed. The python docs don't say anything
> > about 'data_filter' being introduced after 3.0, so can likely
> > assume it always exists.
> 
> According to https://docs.python.org/3/library/tarfile.html :
> 
> "Extraction filters were added to Python 3.12, but may be backported to
> older versions as security updates. To check whether the feature is
> available, use e.g. hasattr(tarfile, 'data_filter') rather than checking the
> Python version."
> 
> And it seems to be missing in Python 3.7, indeed:
> 
>  https://docs.python.org/3.7/library/tarfile.html
> 
> So as long as we still support this old version, I think I've got to keep
> this check.

Opps, yes, I missed the docs. The version info is against the top heading,
not repeated against each method.


With regards,
Daniel