From patchwork Fri Nov 1 21:48:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Whitcroft X-Patchwork-Id: 1188196 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=canonical.com Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 474bRM6HPFz9sPJ; Sat, 2 Nov 2019 08:48:31 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1iQemP-0005X6-Om; Fri, 01 Nov 2019 21:48:25 +0000 Received: from youngberry.canonical.com ([91.189.89.112]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1iQemN-0005WE-KB for kernel-team@lists.ubuntu.com; Fri, 01 Nov 2019 21:48:23 +0000 Received: from 1.general.apw.uk.vpn ([10.172.192.78] helo=localhost) by youngberry.canonical.com with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1iQemN-0002E5-CP; Fri, 01 Nov 2019 21:48:23 +0000 From: Andy Whitcroft To: kernel-team@lists.ubuntu.com Subject: [{B, D, E, F}/master-next 1/2] UBUNTU: [Packaging] dkms -- try launchpad librarian for pool downloads Date: Fri, 1 Nov 2019 21:48:21 +0000 Message-Id: <20191101214822.76610-2-apw@canonical.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191101214822.76610-1-apw@canonical.com> References: <20191101214822.76610-1-apw@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andy Whitcroft Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" When we build a kernel we capture the current dkms package versions. This allows us to keep the versions syncronised in an entire SRU cycle between the master and derivative kernels. It also allows us to maintain the same versions on respin. This suffers from the issue that they may expire from the archive pool if the dkms packages are revved. The same packages are available in the launchpad librarian and indeed (now) via well known names. So when are attempting to download from the archive pool first attempt to get the package from the primary archive librarian first. If this fails continue to do the pool scan as normal. This increases the chance we will find an appropriate package regardless of the currently published packages, whilst retaining the ability to build in a more restricted environment (such as when we have no internet access). BugLink: http://bugs.launchpad.net/bugs/1850958 Signed-off-by: Andy Whitcroft --- debian/scripts/dkms-build | 98 ++++++++++++++++++++++++--------------- 1 file changed, 60 insertions(+), 38 deletions(-) diff --git a/debian/scripts/dkms-build b/debian/scripts/dkms-build index 08c1a8fa1bd4..3c3c28f75c43 100755 --- a/debian/scripts/dkms-build +++ b/debian/scripts/dkms-build @@ -26,56 +26,78 @@ built_using_record() sed -i -e "s/^\(linux:BuiltUsing=.*\)/\1$built_using, /" "$subst" } +# ABI: returns present in $? and located path in lpackage_path when found. +package_present() +{ + for lpackage_path in "$1"_*.deb + do + break + done + [ -f "$lpackage_path" ] +} + # Download and extract the DKMS package -- note there may be more # than one package to install. for package_path in "$@" do - echo "II: dkms-build downloading $package ($(basename $package_path))" + package_file=$(basename "$package_path") + echo "II: dkms-build downloading $package ($package_file)" rpackage=$( echo "$package_path" | sed -e 's@.*/@@' -e 's@_.*@@' ) lpackage=$( echo "$rpackage" | sed -e 's@=.*@@' ) - case "$package_path" in - pool/*) - for pool in $( apt-cache policy | grep '^ [^ ]' | sort -r -n -k 1,1 -s | \ - awk ' - ($2 ~ /^http/) { - if (!($2 in E)) { - E[$2]=1; - print $2; - } - } - ') - do - for lpackage_path in "$lpackage"_*.deb - do - break - done - if [ -f "$lpackage_path" ]; then + while true + do + if package_present "$lpackage"; then + break + fi + case "$package_path" in + pool/*) + # Attempt download from the launchpad librarian first. + wget "https://launchpad.net/ubuntu/+archive/primary/+files/$package_file" || true + if package_present "$lpackage"; then break fi - url="$pool/$package_path" - wget "$url" && break || true - # No components in PPAs. - url=$(echo "$url" | sed -e 's@/pool/[^/]*/@/pool/main/@') - wget "$url" && break || true - done - ;; - http*:*) - wget "$package_path" - ;; - */*) - cp -p "$package_path" . - ;; - *) - apt-get download "$rpackage" - ;; - esac - dpkg -x "$lpackage"_*.deb "$package" - for lpackage_path in "$lpackage"_*.deb - do + # Download from the available pools. + for pool in $( apt-cache policy | grep '^ [^ ]' | sort -r -n -k 1,1 -s | \ + awk ' + ($2 ~ /^http/) { + if (!($2 in E)) { + E[$2]=1; + print $2; + } + } + ') + do + if package_present "$lpackage"; then + break + fi + url="$pool/$package_path" + wget "$url" && break || true + # No components in PPAs. + url=$(echo "$url" | sed -e 's@/pool/[^/]*/@/pool/main/@') + wget "$url" && break || true + done + ;; + http*:*) + wget "$package_path" + ;; + */*) + cp -p "$package_path" . + ;; + *) + apt-get download "$rpackage" + ;; + esac break done + if ! package_present "$lpackage"; then + echo "EE: $lpackage not found" + exit 1 + fi + + dpkg -x "$lpackage"_*.deb "$package" + lversion=$( echo "$lpackage_path" | sed -e 's@.*/@@' -e 's@_[^_]*$@@' -e 's@.*_@@') built_using_record "$srcdir/debian/$pkgname.substvars" "$built_using$lpackage (= $lversion)" done