| Message ID | 20241211191630.1414970-2-bernd@kuhls.net |
|---|---|
| State | Changes Requested |
| Headers | show |
| Series | [1/2] package/python-pyasyncore: enable host package | expand |
Hello Bernd, On Wed, 11 Dec 2024 20:16:30 +0100 Bernd Kuhls <bernd@kuhls.net> wrote: > +PYTHON_EMAILPROXY_VERSION = 2024.11.11 > +PYTHON_EMAILPROXY_SOURCE = emailproxy-$(PYTHON_EMAILPROXY_VERSION).tar.gz > +PYTHON_EMAILPROXY_SITE = https://files.pythonhosted.org/packages/2b/49/386e675f2b0706083fdf02ad3666b98d0875a2e1cdcabff8df9fe641d2e3 > +PYTHON_EMAILPROXY_DEPENDENCIES = host-python-cryptography host-python-pyasyncore I am still not happy with host-python-cryptography host-python-pyasyncore being dependencies, while they definitely shouldn't be needed. I know James asked you to use --skip-dependency-check, and it apparently wasn't working with: File "/home/bernd/buildroot/output/build/python-emailproxy-2024.9.12/ emailproxy.py", line 52, in <module> import asyncore ModuleNotFoundError: No module named 'asyncore' Do we have an explanation why? Adding those dependencies really feels like a hack. Best regards, Thomas
Am 29.12.24 um 22:12 schrieb Thomas Petazzoni via buildroot: > Hello Bernd, > > On Wed, 11 Dec 2024 20:16:30 +0100 > Bernd Kuhls <bernd@kuhls.net> wrote: > >> +PYTHON_EMAILPROXY_VERSION = 2024.11.11 >> +PYTHON_EMAILPROXY_SOURCE = emailproxy-$(PYTHON_EMAILPROXY_VERSION).tar.gz >> +PYTHON_EMAILPROXY_SITE = https://files.pythonhosted.org/packages/2b/49/386e675f2b0706083fdf02ad3666b98d0875a2e1cdcabff8df9fe641d2e3 >> +PYTHON_EMAILPROXY_DEPENDENCIES = host-python-cryptography host-python-pyasyncore > > I am still not happy with host-python-cryptography > host-python-pyasyncore being dependencies, while they definitely > shouldn't be needed. I know James asked you to use > --skip-dependency-check, and it apparently wasn't working with: > > File "/home/bernd/buildroot/output/build/python-emailproxy-2024.9.12/ > emailproxy.py", line 52, in <module> > import asyncore > ModuleNotFoundError: No module named 'asyncore' > > Do we have an explanation why? > > Adding those dependencies really feels like a hack. Looks like the upstream pyproject.toml requires importing the module to read the version: https://github.com/simonrob/email-oauth2-proxy/blob/3ed7dfdcd1d7a9dc2a392aecaae7bb5d913acec0/pyproject.toml#L45 My guess is that cross-build wasn't a consideration there. Might be worth trying to convince upstream to change that, otherwise the question is whether it's preferable for Buildroot to have hackish host dependencies, or a patch to make the version static. Best regards, Fiona
Hi Vincent, Am 13.01.25 um 20:38 schrieb Vincent Fazio: >>>> Am 29.12.24 um 22:12 schrieb Thomas Petazzoni via buildroot: >>>>> I am still not happy with host-python-cryptography host- >>>>> python-pyasyncore being dependencies, while they definitely >>>>> shouldn't be needed. I know James asked you to use --skip- >>>>> dependency-check, and it apparently wasn't working with: > > If we wanted to rely on setuptools' dependency resolution, we need > to remove the `--no-isolation` flag in the build, otherwise it > assumes the dependencies are available within host-python (which is > why the host packages are necessary). I tested dropping `-n` from > the build command and it worked OK. I'm guessing we do this so that > BR is an island and self-contained. thanks for the hint. Yes, removing "-n" from PKG_PYTHON_PEP517_BUILD_CMD fixes the build of python-emailproxy but breaks dozens of other python packages. Adding "-n" to _BUILD_OPTS of these packages fixes them but it will require lots of patches. > If we wanted to avoid the host dependencies, we could just patch the > pyproject to hardcode the version and not rely on setuptools' > dynamic attribute resolution, though it might be best to do this on > the latest tag and not 2024.9.12? Did not test it yet but would such a patch be accepted by buildroot without sending it upstream? Regards, Bernd
On Mon, Jan 27, 2025 at 2:57 AM Bernd Kuhls <bernd@kuhls.net> wrote: > > Hi Vincent, > > Am 13.01.25 um 20:38 schrieb Vincent Fazio: > >>>> Am 29.12.24 um 22:12 schrieb Thomas Petazzoni via buildroot: > >>>>> I am still not happy with host-python-cryptography host- > >>>>> python-pyasyncore being dependencies, while they definitely > >>>>> shouldn't be needed. I know James asked you to use --skip- > >>>>> dependency-check, and it apparently wasn't working with: > > > > If we wanted to rely on setuptools' dependency resolution, we need > > to remove the `--no-isolation` flag in the build, otherwise it > > assumes the dependencies are available within host-python (which is > > why the host packages are necessary). I tested dropping `-n` from > > the build command and it worked OK. I'm guessing we do this so that > > BR is an island and self-contained. > > thanks for the hint. Yes, removing "-n" from PKG_PYTHON_PEP517_BUILD_CMD > fixes the build of python-emailproxy but breaks dozens of other python > packages. Adding "-n" to _BUILD_OPTS of these packages fixes them but it > will require lots of patches. Yeah, so I don't think removing -n is the correct approach here since we want to manage all the build dependencies as part of buildroot. > > > If we wanted to avoid the host dependencies, we could just patch the > > pyproject to hardcode the version and not rely on setuptools' > > dynamic attribute resolution, though it might be best to do this on > > the latest tag and not 2024.9.12? > Did not test it yet but would such a patch be accepted by buildroot > without sending it upstream? Was this a bug in the setuptools ast parser maybe? These dynamic attribute resolvers usually try ast parsing for versions and fallback to traditional imports only if the ast parsing fails. > > Regards, Bernd > > _______________________________________________ > buildroot mailing list > buildroot@buildroot.org > https://lists.buildroot.org/mailman/listinfo/buildroot
> -----Original Message----- > From: James Hilliard <james.hilliard1@gmail.com> > Sent: Sunday, January 26, 2025 7:49 PM > To: Bernd Kuhls <bernd@kuhls.net> > Cc: Vincent Fazio <vfazio@xes-inc.com>; Thomas Petazzoni > <thomas.petazzoni@bootlin.com>; buildroot@buildroot.org; Fiona Klute > <fiona.klute@gmx.de> > Subject: Re: [Buildroot] [External] - Re: [PATCH v2 2/2] package/python- > emailproxy: new package > > On Mon, Jan 27, 2025 at 2:57 AM Bernd Kuhls <bernd@kuhls.net> wrote: > > > > Hi Vincent, > > > > Am 13.01.25 um 20:38 schrieb Vincent Fazio: > > > > > > If we wanted to rely on setuptools' dependency resolution, we need > > > to remove the `--no-isolation` flag in the build, otherwise it > > > assumes the dependencies are available within host-python (which is > > > why the host packages are necessary). I tested dropping `-n` from > > > the build command and it worked OK. I'm guessing we do this so that > > > BR is an island and self-contained. > > > > thanks for the hint. Yes, removing "-n" from > > PKG_PYTHON_PEP517_BUILD_CMD fixes the build of python-emailproxy > but > > breaks dozens of other python packages. Adding "-n" to _BUILD_OPTS of > > these packages fixes them but it will require lots of patches. > > Yeah, so I don't think removing -n is the correct approach here since we want > to manage all the build dependencies as part of buildroot. > Sorry I wasn't clear here, I was not advocating that we remove this, I only did this as a test to prove out that was what was requiring the host dependencies > > > > > If we wanted to avoid the host dependencies, we could just patch the > > > pyproject to hardcode the version and not rely on setuptools' > > > dynamic attribute resolution, though it might be best to do this on > > > the latest tag and not 2024.9.12? > > Did not test it yet but would such a patch be accepted by buildroot > > without sending it upstream? IMO, since this is a requirement of the Buildroot build system, it's fine to add a patch for this with the trailer Upstream: N/A We just have to deal with repatching with every release (hence my suggestion to start with the latest tag) > > Was this a bug in the setuptools ast parser maybe? These dynamic attribute > resolvers usually try ast parsing for versions and fallback to traditional imports > only if the ast parsing fails. I was going to suggest talking to the setuptools group. I haven't looked too closely into this. It would obviously be ideal if it just looked at the AST and didn't try to load everything from the module. > > > > > Regards, Bernd > > > > _______________________________________________ > > buildroot mailing list > > buildroot@buildroot.org > > https://lists.buildroot.org/mailman/listinfo/buildroot
> -----Original Message----- > From: buildroot <buildroot-bounces@buildroot.org> On Behalf Of Vincent > Fazio > Sent: Monday, January 27, 2025 7:10 AM > To: James Hilliard <james.hilliard1@gmail.com>; Bernd Kuhls > <bernd@kuhls.net> > Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>; > buildroot@buildroot.org; Fiona Klute <fiona.klute@gmx.de> > Subject: Re: [Buildroot] [External] - Re: [PATCH v2 2/2] package/python- > emailproxy: new package > > > > > -----Original Message----- > > From: James Hilliard <james.hilliard1@gmail.com> > > Sent: Sunday, January 26, 2025 7:49 PM > > To: Bernd Kuhls <bernd@kuhls.net> > > Cc: Vincent Fazio <vfazio@xes-inc.com>; Thomas Petazzoni > > <thomas.petazzoni@bootlin.com>; buildroot@buildroot.org; Fiona Klute > > <fiona.klute@gmx.de> > > Subject: Re: [Buildroot] [External] - Re: [PATCH v2 2/2] > > package/python- > > emailproxy: new package > > > > On Mon, Jan 27, 2025 at 2:57 AM Bernd Kuhls <bernd@kuhls.net> wrote: > > > > > > Hi Vincent, > > > > > > Am 13.01.25 um 20:38 schrieb Vincent Fazio: > > > > > > > > If we wanted to rely on setuptools' dependency resolution, we need > > > > to remove the `--no-isolation` flag in the build, otherwise it > > > > assumes the dependencies are available within host-python (which > > > > is why the host packages are necessary). I tested dropping `-n` > > > > from the build command and it worked OK. I'm guessing we do this > > > > so that BR is an island and self-contained. > > > > > > thanks for the hint. Yes, removing "-n" from > > > PKG_PYTHON_PEP517_BUILD_CMD fixes the build of python-emailproxy > > but > > > breaks dozens of other python packages. Adding "-n" to _BUILD_OPTS > > > of these packages fixes them but it will require lots of patches. > > > > Yeah, so I don't think removing -n is the correct approach here since > > we want to manage all the build dependencies as part of buildroot. > > > > Sorry I wasn't clear here, I was not advocating that we remove this, I only did > this as a test to prove out that was what was requiring the host dependencies > > > > > > > > If we wanted to avoid the host dependencies, we could just patch > > > > the pyproject to hardcode the version and not rely on setuptools' > > > > dynamic attribute resolution, though it might be best to do this > > > > on the latest tag and not 2024.9.12? > > > Did not test it yet but would such a patch be accepted by buildroot > > > without sending it upstream? > > IMO, since this is a requirement of the Buildroot build system, it's fine to add a > patch for this with the trailer Upstream: N/A > > We just have to deal with repatching with every release (hence my suggestion > to start with the latest tag) > > > > > Was this a bug in the setuptools ast parser maybe? These dynamic > > attribute resolvers usually try ast parsing for versions and fallback > > to traditional imports only if the ast parsing fails. Apparently, part of the problem is that AST parsing expects the attribute to be a fixed string and not "calculated" Traceback (most recent call last): File "/mnt/development/email-oauth2-proxy/venv/lib/python3.10/site-packages/setuptools/config/expand.py", line 191, in read_attr return getattr(StaticModule(module_name, spec), attr_name) File "/mnt/development/email-oauth2-proxy/venv/lib/python3.10/site-packages/setuptools/config/expand.py", line 87, in __getattr__ raise AttributeError(f"{self.name} has no attribute {attr}") from e AttributeError: emailproxy has no attribute __package_version__ Also, the setuptools version requirement is incorrect, the pyproject.toml is defined with features introduced in 62.6.0: https://setuptools.pypa.io/en/stable/history.html#v62-6-0 It could be argued that the upstream package could be "fixed" to not require crypto or asynccore/pyasynccore as a build dep if they just hardcoded the attribute instead of interpreting it from `__version__` so the dynamic attribute could leverage the AST instead of trying to import the module. Their pyproject.toml does complain that it's necessary and implies they do not like this current behavior so may be an easy sell. > > I was going to suggest talking to the setuptools group. I haven't looked too > closely into this. > > It would obviously be ideal if it just looked at the AST and didn't try to load > everything from the module. > > > > > > > > > > > Regards, Bernd > > > > > > _______________________________________________ > > > buildroot mailing list > > > buildroot@buildroot.org > > > https://lists.buildroot.org/mailman/listinfo/buildroot > _______________________________________________ > buildroot mailing list > buildroot@buildroot.org > https://lists.buildroot.org/mailman/listinfo/buildroot
On Mon, Jan 13, 2025 at 8:17 PM Fiona Klute <fiona.klute@gmx.de> wrote: > > Am 29.12.24 um 22:12 schrieb Thomas Petazzoni via buildroot: > > Hello Bernd, > > > > On Wed, 11 Dec 2024 20:16:30 +0100 > > Bernd Kuhls <bernd@kuhls.net> wrote: > > > >> +PYTHON_EMAILPROXY_VERSION = 2024.11.11 > >> +PYTHON_EMAILPROXY_SOURCE = emailproxy-$(PYTHON_EMAILPROXY_VERSION).tar.gz > >> +PYTHON_EMAILPROXY_SITE = https://files.pythonhosted.org/packages/2b/49/386e675f2b0706083fdf02ad3666b98d0875a2e1cdcabff8df9fe641d2e3 > >> +PYTHON_EMAILPROXY_DEPENDENCIES = host-python-cryptography host-python-pyasyncore > > > > I am still not happy with host-python-cryptography > > host-python-pyasyncore being dependencies, while they definitely > > shouldn't be needed. I know James asked you to use > > --skip-dependency-check, and it apparently wasn't working with: > > > > File "/home/bernd/buildroot/output/build/python-emailproxy-2024.9.12/ > > emailproxy.py", line 52, in <module> > > import asyncore > > ModuleNotFoundError: No module named 'asyncore' > > > > Do we have an explanation why? > > > > Adding those dependencies really feels like a hack. > Looks like the upstream pyproject.toml requires importing the module to > read the version: > https://github.com/simonrob/email-oauth2-proxy/blob/3ed7dfdcd1d7a9dc2a392aecaae7bb5d913acec0/pyproject.toml#L45 > > My guess is that cross-build wasn't a consideration there. Might be > worth trying to convince upstream to change that, otherwise the question > is whether it's preferable for Buildroot to have hackish host > dependencies, or a patch to make the version static. I hacked together a quick PR that should fix this: https://github.com/simonrob/email-oauth2-proxy/pull/322 > > Best regards, > Fiona >
diff --git a/DEVELOPERS b/DEVELOPERS index f0006cfa35..532e86acc8 100644 --- a/DEVELOPERS +++ b/DEVELOPERS @@ -482,6 +482,7 @@ F: package/privoxy/ F: package/pure-ftpd/ F: package/python-couchdb/ F: package/python-cssutils/ +F: package/python-emailproxy/ F: package/python-glslang/ F: package/python-mako/ F: package/python-mwclient/ diff --git a/package/Config.in b/package/Config.in index dfae37e1bc..588f3da9c9 100644 --- a/package/Config.in +++ b/package/Config.in @@ -1102,6 +1102,7 @@ menu "External python modules" source "package/python-dpkt/Config.in" source "package/python-dtschema/Config.in" source "package/python-ecdsa/Config.in" + source "package/python-emailproxy/Config.in" source "package/python-email-validator/Config.in" source "package/python-engineio/Config.in" source "package/python-entrypoints/Config.in" diff --git a/package/python-emailproxy/Config.in b/package/python-emailproxy/Config.in new file mode 100644 index 0000000000..f5c72d3a18 --- /dev/null +++ b/package/python-emailproxy/Config.in @@ -0,0 +1,14 @@ +config BR2_PACKAGE_PYTHON_EMAILPROXY + bool "python-emailproxy" + depends on BR2_PACKAGE_HOST_RUSTC_TARGET_ARCH_SUPPORTS # python-cryptography + select BR2_PACKAGE_HOST_PYTHON3 + select BR2_PACKAGE_HOST_PYTHON3_SSL + select BR2_PACKAGE_PYTHON_CRYPTOGRAPHY # runtime + select BR2_PACKAGE_PYTHON_PROMPT_TOOLKIT # runtime + select BR2_PACKAGE_PYTHON_PYASYNCORE # runtime + help + An IMAP/POP/SMTP proxy that transparently adds OAuth 2.0 + authentication for email clients that don't support this + method. + + https://github.com/simonrob/email-oauth2-proxy diff --git a/package/python-emailproxy/python-emailproxy.hash b/package/python-emailproxy/python-emailproxy.hash new file mode 100644 index 0000000000..da3f63639f --- /dev/null +++ b/package/python-emailproxy/python-emailproxy.hash @@ -0,0 +1,5 @@ +# md5, sha256 from https://pypi.org/pypi/emailproxy/json +md5 fa1c82907718c7c643c541606aec947d emailproxy-2024.11.11.tar.gz +sha256 97b59c5f9cd5afb1d2146aff9c5259a3ab0d46b074cb50952b9fb4a4d8b4df35 emailproxy-2024.11.11.tar.gz +# Locally computed sha256 checksums +sha256 c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4 LICENSE diff --git a/package/python-emailproxy/python-emailproxy.mk b/package/python-emailproxy/python-emailproxy.mk new file mode 100644 index 0000000000..58bb1fb7ee --- /dev/null +++ b/package/python-emailproxy/python-emailproxy.mk @@ -0,0 +1,15 @@ +################################################################################ +# +# python-emailproxy +# +################################################################################ + +PYTHON_EMAILPROXY_VERSION = 2024.11.11 +PYTHON_EMAILPROXY_SOURCE = emailproxy-$(PYTHON_EMAILPROXY_VERSION).tar.gz +PYTHON_EMAILPROXY_SITE = https://files.pythonhosted.org/packages/2b/49/386e675f2b0706083fdf02ad3666b98d0875a2e1cdcabff8df9fe641d2e3 +PYTHON_EMAILPROXY_DEPENDENCIES = host-python-cryptography host-python-pyasyncore +PYTHON_EMAILPROXY_SETUP_TYPE = setuptools +PYTHON_EMAILPROXY_LICENSE = Apache-2.0 +PYTHON_EMAILPROXY_LICENSE_FILES = LICENSE + +$(eval $(python-package))
Signed-off-by: Bernd Kuhls <bernd@kuhls.net> --- v2: bump version to 2024.11.11 DEVELOPERS | 1 + package/Config.in | 1 + package/python-emailproxy/Config.in | 14 ++++++++++++++ package/python-emailproxy/python-emailproxy.hash | 5 +++++ package/python-emailproxy/python-emailproxy.mk | 15 +++++++++++++++ 5 files changed, 36 insertions(+) create mode 100644 package/python-emailproxy/Config.in create mode 100644 package/python-emailproxy/python-emailproxy.hash create mode 100644 package/python-emailproxy/python-emailproxy.mk