From patchwork Thu Feb 21 16:54:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thadeu Lima de Souza Cascardo X-Patchwork-Id: 1046289 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4450vS5wfrz9sBF; Fri, 22 Feb 2019 03:55:00 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1gwrcd-0005Hq-Gq; Thu, 21 Feb 2019 16:54:55 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:128) (Exim 4.86_2) (envelope-from ) id 1gwrcb-0005HA-FU for kernel-team@lists.ubuntu.com; Thu, 21 Feb 2019 16:54:53 +0000 Received: from [194.204.107.10] (helo=localhost.localdomain) by youngberry.canonical.com with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.76) (envelope-from ) id 1gwrcb-0001IL-4Y for kernel-team@lists.ubuntu.com; Thu, 21 Feb 2019 16:54:53 +0000 From: Thadeu Lima de Souza Cascardo To: kernel-team@lists.ubuntu.com Subject: [trusty/azure 1/3] UBUNTU: Packaging: Introduce copy-files and local-mangle Date: Thu, 21 Feb 2019 13:54:49 -0300 Message-Id: <20190221165451.4232-2-cascardo@canonical.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190221165451.4232-1-cascardo@canonical.com> References: <20190221165451.4232-1-cascardo@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" BugLink: https://bugs.launchpad.net/bugs/1786013 Two new scripts are introduced in order to do some of the copying and mangling of copies that update-from-*master does. One of the changes on copy-files compared to update-from-*master is that the -c option is given to rsync, so it compares checksums of files in order to decide whether they are different and need an update. That's necessary because sometimes files will have the same size and their modified time will be whithin one second or the original file will be older because git checked it out earlier. The script is split in two so the copy-files may be shared between different kernel tress and the very specific changes are done on the local-mangle file, which is different between trees. Also, in order to make the copy-files the same one for all trees, some of the copies and updates are dependent on a local.conf file, which is present only on those trees where it's needed. The contents of those files are not so easily generated, so they are not part of update.conf. Signed-off-by: Thadeu Lima de Souza Cascardo --- .../scripts/helpers/copy-files | 67 +++++++++++++++++++ .../scripts/helpers/local-mangle | 13 ++++ 2 files changed, 80 insertions(+) create mode 100755 debian.azure-trusty/scripts/helpers/copy-files create mode 100755 debian.azure-trusty/scripts/helpers/local-mangle diff --git a/debian.azure-trusty/scripts/helpers/copy-files b/debian.azure-trusty/scripts/helpers/copy-files new file mode 100755 index 000000000000..0ce0afe84578 --- /dev/null +++ b/debian.azure-trusty/scripts/helpers/copy-files @@ -0,0 +1,67 @@ +#!/bin/bash -eu + +if [ -f debian/debian.env ]; then + # shellcheck disable=SC1091 + . debian/debian.env +fi + +if [ ! -d "${DEBIAN}" ]; then + echo You must run this script from the top directory of this repository. + exit 1 +fi + +CONF="${DEBIAN}"/etc/update.conf +if [ -f "${CONF}" ]; then + # shellcheck disable=SC1090 + . "${CONF}" +fi + +FOREIGN_ARCHES="" +LOCAL_CONF="${DEBIAN}/etc/local.conf" +if [ -f "${LOCAL_CONF}" ]; then + # shellcheck disable=SC1090 + . "${LOCAL_CONF}" +fi + +SKIP_RULES_D=${SKIP_RULES_D:-} + +# +# Pick up any master branch changes to udeb modules or firmware. +# +rsync -avc --delete "${DEBIAN_MASTER}/d-i/" "${DEBIAN}/d-i" + +# +# Update configs from master +# +rsync -avc --delete "${DEBIAN_MASTER}/config/" "${DEBIAN}/config" + +# +# Update package and DTB settings from master. +# +if [ -z "${SKIP_RULES_D}" ] ; then + rsync -avc "${DEBIAN_MASTER}/rules.d/"*.mk "${DEBIAN}/rules.d/" +fi + +# Remove the .mk files from the arch's that are not supported +for i in ${FOREIGN_ARCHES} +do + rm -f "${DEBIAN}/rules.d/${i}.mk" + git rm -f --ignore-unmatch "${DEBIAN}/rules.d/${i}.mk" || true +done + +# +# Update modprobe.d from master +# +# Some releases (trusty) don't have this directory, and rsync would fail +# without this check. +if [ -d "${DEBIAN}/modprobe.d/" ]; then + rsync -avc --delete "${DEBIAN_MASTER}/modprobe.d/" "${DEBIAN}/modprobe.d" +fi + +cp -p "${DEBIAN_MASTER}/control.d/"*.inclusion-list "${DEBIAN}/control.d" + +cp -p "${DEBIAN_MASTER}/reconstruct" "${DEBIAN}/reconstruct" + +if [ -x "${DEBIAN}/scripts/helpers/local-mangle" ]; then + "./${DEBIAN}/scripts/helpers/local-mangle" +fi diff --git a/debian.azure-trusty/scripts/helpers/local-mangle b/debian.azure-trusty/scripts/helpers/local-mangle new file mode 100755 index 000000000000..daa2f1f55d74 --- /dev/null +++ b/debian.azure-trusty/scripts/helpers/local-mangle @@ -0,0 +1,13 @@ +#!/bin/bash -eu + +# shellcheck disable=SC1091 +. debian/debian.env + +# +# Turn off strong stack protection as it requires gcc 4.9+ +# +sed -i -e 's/^.*CONFIG_CC_STACKPROTECTOR_REGULAR.*$/CONFIG_CC_STACKPROTECTOR_REGULAR=y/' -e 's/^CONFIG_CC_STACKPROTECTOR_STRONG.*$/# CONFIG_CC_STACKPROTECTOR_STRONG is not set/' "${DEBIAN}/config/config.common.ubuntu" +sed -i 's/CONFIG_CC_STACKPROTECTOR_STRONG/CONFIG_CC_STACKPROTECTOR_REGULAR/' "${DEBIAN}/config/annotations" + +# Disable ZFS as we have no tooling in trusty to mount these. +sed -i -e 's/^\(do_zfs[ \t]*=[ \t]\)true\(.*\)$/\1false\2/' "${DEBIAN}/rules.d/"*.mk From patchwork Thu Feb 21 16:54:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thadeu Lima de Souza Cascardo X-Patchwork-Id: 1046291 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4450vT3G8cz9sDL; Fri, 22 Feb 2019 03:55:01 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1gwrcd-0005I3-Lp; Thu, 21 Feb 2019 16:54:55 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:128) (Exim 4.86_2) (envelope-from ) id 1gwrcb-0005HG-Ow for kernel-team@lists.ubuntu.com; Thu, 21 Feb 2019 16:54:53 +0000 Received: from [194.204.107.10] (helo=localhost.localdomain) by youngberry.canonical.com with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.76) (envelope-from ) id 1gwrcb-0001IL-E7 for kernel-team@lists.ubuntu.com; Thu, 21 Feb 2019 16:54:53 +0000 From: Thadeu Lima de Souza Cascardo To: kernel-team@lists.ubuntu.com Subject: [trusty/azure 2/3] UBUNTU: Packaging: Make update-from-*master call copy-files Date: Thu, 21 Feb 2019 13:54:50 -0300 Message-Id: <20190221165451.4232-3-cascardo@canonical.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190221165451.4232-1-cascardo@canonical.com> References: <20190221165451.4232-1-cascardo@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" BugLink: https://bugs.launchpad.net/bugs/1786013 Now that copy-files and local-magle are introduced, update-from-*master may make use of them, instead of doing the copies and mangles itself. That makes it easier to replace update-from-*master with a single script version for all trees in the future. Signed-off-by: Thadeu Lima de Souza Cascardo --- .../etc/update-from-azure-xenial-master | 32 +------------------ .../scripts/helpers/local-mangle | 2 +- 2 files changed, 2 insertions(+), 32 deletions(-) diff --git a/debian.azure-trusty/etc/update-from-azure-xenial-master b/debian.azure-trusty/etc/update-from-azure-xenial-master index 5a819c7e4a09..98de164a3bf9 100755 --- a/debian.azure-trusty/etc/update-from-azure-xenial-master +++ b/debian.azure-trusty/etc/update-from-azure-xenial-master @@ -96,37 +96,7 @@ then exit 1 fi -# -# Pick up any master branch changes to udeb modules or firmware. -# -rsync -av --delete ${DEBIAN_SOURCE}/d-i/ ${DEBIAN_TARGET}/d-i - -# -# Update configs from master -# -rsync -av --delete ${DEBIAN_SOURCE}/config/ ${DEBIAN_TARGET}/config - -# -# Turn off strong stack protection as it requires gcc 4.9+ -# -sed -i -e 's/^.*CONFIG_CC_STACKPROTECTOR_REGULAR.*$/CONFIG_CC_STACKPROTECTOR_REGULAR=y/' -e 's/^CONFIG_CC_STACKPROTECTOR_STRONG.*$/# CONFIG_CC_STACKPROTECTOR_STRONG is not set/' ${DEBIAN_TARGET}/config/config.common.ubuntu -sed -i 's/CONFIG_CC_STACKPROTECTOR_STRONG/CONFIG_CC_STACKPROTECTOR_REGULAR/' ${DEBIAN_TARGET}/config/annotations - -# -# Update package and DTB settings from master. -# -rsync -av ${DEBIAN_SOURCE}/rules.d/*.mk ${DEBIAN_TARGET}/rules.d/ -# Disable ZFS as we have no tooling in trusty to mount these. -sed -i -e 's/^\(do_zfs[ \t]*=[ \t]\)true\(.*\)$/\1false\2/' debian.xenial/rules.d/*.mk - -# -# Update modprobe.d from master -# -rsync -av --delete ${DEBIAN_SOURCE}/modprobe.d/ ${DEBIAN_TARGET}/modprobe.d - -cp -p ${DEBIAN_SOURCE}/control.d/*.inclusion-list ${DEBIAN_TARGET}/control.d - -cp -p ${DEBIAN_SOURCE}/reconstruct ${DEBIAN_TARGET}/reconstruct +"./${DEBIAN_TARGET}/scripts/helpers/copy-files" fakeroot debian/rules clean updateconfigs diff --git a/debian.azure-trusty/scripts/helpers/local-mangle b/debian.azure-trusty/scripts/helpers/local-mangle index daa2f1f55d74..79f36e1b114f 100755 --- a/debian.azure-trusty/scripts/helpers/local-mangle +++ b/debian.azure-trusty/scripts/helpers/local-mangle @@ -10,4 +10,4 @@ sed -i -e 's/^.*CONFIG_CC_STACKPROTECTOR_REGULAR.*$/CONFIG_CC_STACKPROTECTOR_REG sed -i 's/CONFIG_CC_STACKPROTECTOR_STRONG/CONFIG_CC_STACKPROTECTOR_REGULAR/' "${DEBIAN}/config/annotations" # Disable ZFS as we have no tooling in trusty to mount these. -sed -i -e 's/^\(do_zfs[ \t]*=[ \t]\)true\(.*\)$/\1false\2/' "${DEBIAN}/rules.d/"*.mk +sed -i -e 's/^\(do_zfs[ \t]*=[ \t]\)true\(.*\)$/\1false\2/' "debian.xenial/rules.d/"*.mk From patchwork Thu Feb 21 16:54:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thadeu Lima de Souza Cascardo X-Patchwork-Id: 1046292 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4450vX4gQMz9sBF; Fri, 22 Feb 2019 03:55:04 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1gwrch-0005LD-7z; Thu, 21 Feb 2019 16:54:59 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:128) (Exim 4.86_2) (envelope-from ) id 1gwrcc-0005HR-Aw for kernel-team@lists.ubuntu.com; Thu, 21 Feb 2019 16:54:54 +0000 Received: from [194.204.107.10] (helo=localhost.localdomain) by youngberry.canonical.com with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.76) (envelope-from ) id 1gwrcc-0001IL-05 for kernel-team@lists.ubuntu.com; Thu, 21 Feb 2019 16:54:54 +0000 From: Thadeu Lima de Souza Cascardo To: kernel-team@lists.ubuntu.com Subject: [trusty/azure 3/3] UBUNTU: Packaging: Really disable ZFS Date: Thu, 21 Feb 2019 13:54:51 -0300 Message-Id: <20190221165451.4232-4-cascardo@canonical.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190221165451.4232-1-cascardo@canonical.com> References: <20190221165451.4232-1-cascardo@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" The update script was supposed to disable ZFS, but it wasn't. Now that it was split into copy-file and local-mangle, fix local-mangle so it will really disable it. Signed-off-by: Thadeu Lima de Souza Cascardo --- debian.azure-trusty/scripts/helpers/local-mangle | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/debian.azure-trusty/scripts/helpers/local-mangle b/debian.azure-trusty/scripts/helpers/local-mangle index 79f36e1b114f..daa2f1f55d74 100755 --- a/debian.azure-trusty/scripts/helpers/local-mangle +++ b/debian.azure-trusty/scripts/helpers/local-mangle @@ -10,4 +10,4 @@ sed -i -e 's/^.*CONFIG_CC_STACKPROTECTOR_REGULAR.*$/CONFIG_CC_STACKPROTECTOR_REG sed -i 's/CONFIG_CC_STACKPROTECTOR_STRONG/CONFIG_CC_STACKPROTECTOR_REGULAR/' "${DEBIAN}/config/annotations" # Disable ZFS as we have no tooling in trusty to mount these. -sed -i -e 's/^\(do_zfs[ \t]*=[ \t]\)true\(.*\)$/\1false\2/' "debian.xenial/rules.d/"*.mk +sed -i -e 's/^\(do_zfs[ \t]*=[ \t]\)true\(.*\)$/\1false\2/' "${DEBIAN}/rules.d/"*.mk