From patchwork Fri Nov 18 09:10:21 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?SsOpcsO0bWUgUG91aWxsZXI=?= X-Patchwork-Id: 696509 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3tKsgt1X60z9t2D for ; Fri, 18 Nov 2016 20:11:54 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 3CFED2EC27; Fri, 18 Nov 2016 09:11:52 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qWzrXp5U+EsZ; Fri, 18 Nov 2016 09:11:42 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id 1BD912EE90; Fri, 18 Nov 2016 09:11:07 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by ash.osuosl.org (Postfix) with ESMTP id 8DBB51C067B for ; Fri, 18 Nov 2016 09:10:31 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id 88CB48B65D for ; Fri, 18 Nov 2016 09:10:31 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CW7e-M7JvCjq for ; Fri, 18 Nov 2016 09:10:30 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from lupi.sysmic.org (sysmic.org [62.210.89.17]) by fraxinus.osuosl.org (Postfix) with ESMTPS id 2ED32883BC for ; Fri, 18 Nov 2016 09:10:30 +0000 (UTC) Received: from lupi.online.net (sysmic.org [62.210.89.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: jezz) by lupi.sysmic.org (Postfix) with ESMTPSA id 5E1334272E; Fri, 18 Nov 2016 10:10:27 +0100 (CET) From: =?UTF-8?q?J=C3=A9r=C3=B4me=20Pouiller?= To: buildroot@busybox.net Date: Fri, 18 Nov 2016 10:10:21 +0100 Message-Id: <1479460224-6119-13-git-send-email-jezz@sysmic.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1479460224-6119-1-git-send-email-jezz@sysmic.org> References: <1479460224-6119-1-git-send-email-jezz@sysmic.org> MIME-Version: 1.0 Cc: Thomas Petazzoni , =?UTF-8?q?J=C3=A9r=C3=B4me=20Pouiller?= Subject: [Buildroot] [PATCH v2 12/15] python3: generate reproducible .pyc X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" .pyc files contain modification time of .py source. In order to make build reproducible, we fix modification time of all .py before to compile .pyc files. In order to guarantee .pyc are regenerated regardless their modification time, we remove .pyc before to compile. However, I wonder if it wouldn't be simpler to always call compile_all with 'force' flag. This work was sponsored by `BA Robotic Systems'. Signed-off-by: Jérôme Pouiller --- package/python3/python3.mk | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-) diff --git a/package/python3/python3.mk b/package/python3/python3.mk index b3f31c0..158c29c 100644 --- a/package/python3/python3.mk +++ b/package/python3/python3.mk @@ -219,6 +219,25 @@ PYTHON3_PATH = $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR)/sysconfigdat $(eval $(autotools-package)) $(eval $(host-autotools-package)) +# Normally, *.pyc files should not have been compiled, but just in +# case, we make sure we remove all of them. +ifeq ($(BR2_PACKAGE_PYTHON3_PY_ONLY),y) +define PYTHON3_REMOVE_PYC_FILES + find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR) -name '*.py' -print0 | \ + sed -z -e s/py$$/pyc/ | \ + xargs -0 --no-run-if-empty rm -f +endef +PYTHON3_TARGET_FINALIZE_HOOKS += PYTHON3_REMOVE_PYC_FILES +endif + +ifeq ($(BR2_REPRODUCIBLE),y) +define PYTHON3_FIX_TIME + find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR) -name '*.py' -print0 | \ + xargs -0 --no-run-if-empty touch -d @$(SOURCE_DATE_EPOCH) +endef +PYTHON3_TARGET_FINALIZE_HOOKS += PYTHON3_FIX_TIME +endif + define PYTHON3_CREATE_PYC_FILES PYTHONPATH="$(PYTHON3_PATH)" \ $(HOST_DIR)/usr/bin/python$(PYTHON3_VERSION_MAJOR) \ @@ -238,16 +257,6 @@ endef PYTHON3_TARGET_FINALIZE_HOOKS += PYTHON3_REMOVE_PY_FILES endif -# Normally, *.pyc files should not have been compiled, but just in -# case, we make sure we remove all of them. -ifeq ($(BR2_PACKAGE_PYTHON3_PY_ONLY),y) -define PYTHON3_REMOVE_PYC_FILES - find $(TARGET_DIR)/usr/lib/python$(PYTHON3_VERSION_MAJOR) -name '*.pyc' -print0 | \ - xargs -0 --no-run-if-empty rm -f -endef -PYTHON3_TARGET_FINALIZE_HOOKS += PYTHON3_REMOVE_PYC_FILES -endif - # In all cases, we don't want to keep the optimized .opt-1.pyc and # .opt-2.pyc files, since they can't work without their non-optimized # variant.