From patchwork Sat Jun 27 11:44:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guillaume Bres X-Patchwork-Id: 1318305 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=busybox.net (client-ip=140.211.166.133; helo=hemlock.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=TROBQQcC; dkim-atps=neutral Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 49vBkH2QP9z9sRW for ; Sat, 27 Jun 2020 21:44:38 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id 6FAF28873F; Sat, 27 Jun 2020 11:44:35 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8o4xKk9tFT0i; Sat, 27 Jun 2020 11:44:34 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by hemlock.osuosl.org (Postfix) with ESMTP id 1A9DC88703; Sat, 27 Jun 2020 11:44:34 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 8C0E71BF44A for ; Sat, 27 Jun 2020 11:44:31 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 85A2787C42 for ; Sat, 27 Jun 2020 11:44:31 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id voNqXxwm6Ofc for ; Sat, 27 Jun 2020 11:44:30 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wr1-f65.google.com (mail-wr1-f65.google.com [209.85.221.65]) by whitealder.osuosl.org (Postfix) with ESMTPS id 36F8B879FD for ; Sat, 27 Jun 2020 11:44:30 +0000 (UTC) Received: by mail-wr1-f65.google.com with SMTP id h5so11928144wrc.7 for ; Sat, 27 Jun 2020 04:44:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pgwcEuiPtcmHz4TbNLTW9NZWQ9j5oq8McnAnbIFcsKY=; b=TROBQQcCaztBlJXJkVSO3nblrBGxcVrEBGnxjr5CWxltdht30UCvQkZPWwpIOTdaMJ LGpUgXz0UjCYPz0+1izpnq7/wZBozQKTTAP+UliMH/4/nPg0e4uR65toNjdZyMCwT8Vj 0tvBctGrPGmJQGqsqQyPLeRXJLgslmprGullZzWO6Pu4W/b4TwKMl6Lh5IihQ/DVeS6U q+OC2/BbKFoiJl7aXMyZxJhKjHKqt+7GIyKx9e6hWul6cEb93IB+PxrqSw+IUVmX26Df 0nLp4+QtQNXGyKvZHyrixwPJxbfxUyrMsEQu8hrtLIgJLUCCg4oTjy7nAKH2GXwUO8LK 2qKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pgwcEuiPtcmHz4TbNLTW9NZWQ9j5oq8McnAnbIFcsKY=; b=l9znNvt843fwEWtoJllnVRR17AVxc/BCSLmQHQeg2k7HwD/k/6Yctp5pQME81p2Xv6 RndCMUEAnuJQMxCD7LssQJ0X3ZjZclqaO+p2XaaNx71i4EvCYCfd+dqEF72KodPhayF3 Qe62CdErupwFMFr3OiwbiUKgqWXlPbmKlW/Qm+kwesrfwQ1Xp1ifXEdpU8ECVj120ia5 WmzHRpfDUyEfNXEn0qHwkLv9R3LVPfTBe5Rlox112Zo7FbEY6ySmx0ygjxWHcigXg2nB Qbmu8iOHJAqgfZsFqfybOV5sopEgc2wPx0l31Ge52JTYFm/rEQv+F26c7O/GYlamdWC9 IupA== X-Gm-Message-State: AOAM531QmGsskzSzhp5ZWfctVQY43SKmBWLva/60uvhzypMTK4GPihn6 xa0VW2ZrPXi0avUsZKjVUTLPcRJA X-Google-Smtp-Source: ABdhPJxffdVHaLOEYPAzX1JMz0kb6DVyVFhQGLLjhgdH2F1DSAZVTws2oB6+IT8uRCVlGa6oc1qOIw== X-Received: by 2002:a5d:40ca:: with SMTP id b10mr8919719wrq.56.1593258268400; Sat, 27 Jun 2020 04:44:28 -0700 (PDT) Received: from pc-140.home ([2a01:cb1c:794:2300:d109:33e7:bc94:49e5]) by smtp.gmail.com with ESMTPSA id u23sm27808944wru.94.2020.06.27.04.44.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 27 Jun 2020 04:44:27 -0700 (PDT) From: guillaume.bressaix@gmail.com To: buildroot@busybox.net Date: Sat, 27 Jun 2020 13:44:03 +0200 Message-Id: <20200627114403.25017-3-guillaume.bressaix@gmail.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200627114403.25017-1-guillaume.bressaix@gmail.com> References: <20200627114403.25017-1-guillaume.bressaix@gmail.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH 3/3] package/python-scipy: new package X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Guillaume W. Bres" Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" From: "Guillaume W. Bres" Scipy is a powerful python library built on top of Numpy, for statictics, equations solving & system optimization. depends on BR2_TOOLCHAIN_HAS_FORTRAN: unlike python-numpy, this is not an option. Unfortunately we also must pass 'F90=$(TARGET_FC)' to configure step, for gfortran to be known (see down below for precise error messages). python-numpy disregards BR2_TOOLCHAIN_HAS_FORTRAN, I can confirm scipy does not require numpy to be compiled using gfortran to work properly. depends on BR2_PACKAGE_PYTHON_NUMPY_ARCH_SUPPORTS: I followed what is done in python-matplolib for this topic, I should be close what you expect. select BR2_PACKAGE_CLAPACK + OPENBLAS Unlike python-numpy, these libraries are not an option. PYTHON_SCIPY_DEPENDENCIES: current ordering will have numpy use clapack & openblas, but I can confirm it does not matter to have numpy use these libraries for scipy to work fine. PYTHON_SCIPY_DEPENDENCIES += clapack & openblas is not enough. In this setup, 'configure' fails when searching for those packages. See down below for exact error message. The only way I found to pass this step is to customize the 'site.cfg' file, like it is done in python-numpy.mk for instance. Technicallly, host-python-cython is also a requirement (see setup.py). But it is in host-python-numpy list of requirement. Therefore !! I did not put it in PYTHON_SCIPY_DEPENDECIES !! please correct me on this Signed-off-by: Guillaume W. Bres --- All my tests were made using the current raspberrypi3 & raspberrypi0w defconfig environments. I am not able to use the ./util-test-pkg against a python-numpy requirement, maybe I'm missing something here, i would have liked to test this against several glibc/musl toolchains. This is what we get if F90=$(TARGET_FC) is not passed to configure: build fails half way through, even though gfortran does exists, it's not able to find it: building 'mach' library error: library mach has Fortran sources but no Fortran compiler found This is what we get if we remove the 'site.cfg' customization, that is, even though clapack & blas have already been installed, setup.py is not able to locate them. numpy.distutils.system_info.NotFoundError: No lapack/blas resources found. This is the linker error message we get if we remove the extra linker flag pointing to numpy staging dir: [..]/buildroot/output/host/lib/gcc/arm-buildroot-linux-gnueabihf/8.4.0/../../../../arm-buildroot-linux-gnueabihf/bin/ld: [..]/buildroot/output/host/lib/python3.8/site-packages/numpy/core/lib/libnpymath.a: error adding symbols: file format not recognized collect2: error: ld returned 1 exit status --- DEVELOPERS | 1 + package/Config.in | 1 + package/python-scipy/Config.in | 24 ++++++++++++++++++++ package/python-scipy/python-scipy.hash | 4 ++++ package/python-scipy/python-scipy.mk | 31 ++++++++++++++++++++++++++ 5 files changed, 61 insertions(+) create mode 100644 package/python-scipy/Config.in create mode 100644 package/python-scipy/python-scipy.hash create mode 100644 package/python-scipy/python-scipy.mk diff --git a/DEVELOPERS b/DEVELOPERS index 4b6a346a05..c26cb813e5 100644 --- a/DEVELOPERS +++ b/DEVELOPERS @@ -1058,6 +1058,7 @@ F: package/libnids/ F: package/liquid-dsp/ F: package/pixiewps/ F: package/python-pybind/ +F: package/python-scipy/ F: package/reaver/ N: Guo Ren diff --git a/package/Config.in b/package/Config.in index 6a34a895af..1bb2876867 100644 --- a/package/Config.in +++ b/package/Config.in @@ -1134,6 +1134,7 @@ menu "External python modules" source "package/python-scandir/Config.in" source "package/python-scapy/Config.in" source "package/python-schedule/Config.in" + source "package/python-scipy/Config.in" source "package/python-sdnotify/Config.in" source "package/python-secretstorage/Config.in" source "package/python-see/Config.in" diff --git a/package/python-scipy/Config.in b/package/python-scipy/Config.in new file mode 100644 index 0000000000..a9d9fa181e --- /dev/null +++ b/package/python-scipy/Config.in @@ -0,0 +1,24 @@ +config BR2_PACKAGE_PYTHON_SCIPY + bool "python-scipy" + depends on BR2_TOOLCHAIN_HAS_FORTRAN + depends on BR2_PACKAGE_OPENBLAS_ARCH_SUPPORTS # build + depends on BR2_PACKAGE_PYTHON_NUMPY_ARCH_SUPPORTS # python-numpy + depends on BR2_TOOLCHAIN_USES_GLIBC || BR2_TOOLCHAIN_USES_MUSL # python-numpy + select BR2_PACKAGE_OPENBLAS # build + select BR2_PACKAGE_CLAPACK # build + select BR2_PACKAGE_PYTHON_NUMPY # runtime + help + Scipy is a Python package that includes modules for + statistics, optimization, linear algebra and much + more. + + http://www.scipy.org/ + +comment "python-scipy needs toolchain with fortran" + depends on !BR2_TOOLCHAIN_HAS_FORTRAN + +comment "python-scipy needs lib openblas" + depends on !BR2_PACKAGE_OPENBLAS_ARCH_SUPPORTS + +comment "python-scipy needs python-numpy" + depends on !BR2_PACKAGE_PYTHON_NUMPY_ARCH_SUPPORTS diff --git a/package/python-scipy/python-scipy.hash b/package/python-scipy/python-scipy.hash new file mode 100644 index 0000000000..22075fb85e --- /dev/null +++ b/package/python-scipy/python-scipy.hash @@ -0,0 +1,4 @@ +# Locally calculated +sha256 fd8c7b907c3fc4df1830cdce1608057248deb30afa34ac4f8f31c3784b3e4e53 v1.4.1.tar.gz +# License files, locally calculated +sha256 0615c3b553439d39155885fedd7078a923974e84330f8d5d7660074e7c8826a3 LICENSE.txt diff --git a/package/python-scipy/python-scipy.mk b/package/python-scipy/python-scipy.mk new file mode 100644 index 0000000000..46de1993a8 --- /dev/null +++ b/package/python-scipy/python-scipy.mk @@ -0,0 +1,31 @@ +################################################################################ +# +# python-scipy +# +################################################################################ + +PYTHON_SCIPY_VERSION = 1.4.1 +PYTHON_SCIPY_SOURCE = v$(PYTHON_SCIPY_VERSION).tar.gz +PYTHON_SCIPY_SITE = https://github.com/scipy/scipy/archive +PYTHON_SCIPY_LICENSE = BSD-3-Clause +PYTHON_SCIPY_LICENSE_FILES = LICENSE.txt doc/sphinxext/LICENSE.txt \ + doc/scipy-sphinx-theme/LICENSE.txt +PYTHON_SCIPY_SETUP_TYPE = setuptools +PYTHON_SCIPY_DEPENDENCIES = clapack openblas host-python-numpy \ + host-python-pip host-python-pybind + +PYTHON_SCIPY_ENV += LDFLAGS="$(TARGET_LDFLAGS) -shared \ + -L$(PYTHON3_PATH)/site-packages/numpy/core/lib" + +# must be used to locate 'gfortran' +PYTHON_SCIPY_ENV += F90="$(TARGET_FC)" + +# trick to locate 'lapack' and 'blas' +define PYTHON_SCIPY_CONFIGURE_CMDS + rm -f $(@D)/site.cfg + echo "[DEFAULT]" >> $(@D)/site.cfg + echo "library_dirs = $(STAGING_DIR)/usr/lib" >> $(@D)/site.cfg + echo "include_dirs = $(STAGING_DIR)/usr/include" >> $(@D)/site.cfg +endef + +$(eval $(python-package))