From patchwork Thu Nov 17 04:15:23 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sam Bobroff X-Patchwork-Id: 695983 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3tK78j29Wjz9t2n for ; Thu, 17 Nov 2016 15:15:48 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id B45CC25DF4; Thu, 17 Nov 2016 04:15:40 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rJSeO+iDAlOC; Thu, 17 Nov 2016 04:15:34 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id 2F3D525DEC; Thu, 17 Nov 2016 04:15:34 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 06CC01C048D for ; Thu, 17 Nov 2016 04:15:32 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 03AAD864CB for ; Thu, 17 Nov 2016 04:15:32 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fyZ7kTMDQveY for ; Thu, 17 Nov 2016 04:15:31 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by whitealder.osuosl.org (Postfix) with ESMTPS id 2C539864BC for ; Thu, 17 Nov 2016 04:15:31 +0000 (UTC) Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.17/8.16.0.17) with SMTP id uAH4E9gG078653 for ; Wed, 16 Nov 2016 23:15:30 -0500 Received: from e23smtp09.au.ibm.com (e23smtp09.au.ibm.com [202.81.31.142]) by mx0a-001b2d01.pphosted.com with ESMTP id 26s2empav9-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Wed, 16 Nov 2016 23:15:30 -0500 Received: from localhost by e23smtp09.au.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 17 Nov 2016 14:15:28 +1000 Received: from d23dlp02.au.ibm.com (202.81.31.213) by e23smtp09.au.ibm.com (202.81.31.206) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 17 Nov 2016 14:15:25 +1000 Received: from d23relay10.au.ibm.com (d23relay10.au.ibm.com [9.190.26.77]) by d23dlp02.au.ibm.com (Postfix) with ESMTP id E41322BB0059 for ; Thu, 17 Nov 2016 15:15:24 +1100 (EST) Received: from d23av03.au.ibm.com (d23av03.au.ibm.com [9.190.234.97]) by d23relay10.au.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id uAH4FOTd45285556 for ; Thu, 17 Nov 2016 15:15:24 +1100 Received: from d23av03.au.ibm.com (localhost [127.0.0.1]) by d23av03.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id uAH4FOTg022190 for ; Thu, 17 Nov 2016 15:15:24 +1100 Received: from ozlabs.au.ibm.com (ozlabs.au.ibm.com [9.192.253.14]) by d23av03.au.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id uAH4FOiR022183 for ; Thu, 17 Nov 2016 15:15:24 +1100 Received: from tungsten.ozlabs.ibm.com (haven.au.ibm.com [9.192.254.114]) (using TLSv1.2 with cipher AES128-SHA256 (128/128 bits)) (No client certificate requested) by ozlabs.au.ibm.com (Postfix) with ESMTPSA id 1DDE1A010A for ; Thu, 17 Nov 2016 15:15:24 +1100 (AEDT) From: Sam Bobroff To: buildroot@busybox.net Date: Thu, 17 Nov 2016 15:15:23 +1100 X-Mailer: git-send-email 2.1.4 X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 16111704-0052-0000-0000-000001ED5625 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 16111704-0053-0000-0000-0000073AD73E Message-Id: <56063c394c473bf4292658511bae0dd99583be07.1479356115.git.sam.bobroff@au1.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2016-11-17_02:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=1 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1609300000 definitions=main-1611170079 Subject: [Buildroot] [RFC PATCH 1/1] pkg-autotools: generic configure fix for powerpc64 X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Many (100+) packages supported by buildroot contain old configure scripts (or build them from old versions of autotools) that are unable to determine how to link shared libraries on powerpc64 and powerpc64le. This causes that test to erroneously fail on toolchains that are not "bi-endian" (which is the case for toolchains built by buildroot), which causes configure to build static libraries instead of dynamic ones. Although these builds succeed, they tend to cause linker failures in binaries later linked against them. Because affected configure files can be discovered automatically, this patch introduces a hook (enabled only when building for powerpc64 and powerpc64le) that uses a script to scan and fix each package. This applies only to packages built for the target. Signed-off-by: Sam Bobroff --- Hi Buildroot, See the patch commit message for a description of the issue I'm trying to fix. This is an RFC and here are some things to consider: I took the approach of automatically patching packages. This presumably slows the build down a little bit, but only on the affacted architectures (the code is not enabled otherwise) and has the advantage of automatically picking up new packages as they are added. I've added a minimum of code to pkg-autotools.mk to add a PRE_CONFIGURE hook to the end of the existing hooks. It's important that it be after the AUTO_RECONFIGURE handling, which may regenerate configure. The actual patching work is done in a support script. The script is fairly simple: affected configure scripts all contain a unique pattern, so files are selected based on that, and "patch" is used to correct them. The location of the patch moves around a lot but there are lots of constant lines of context around them so patch always succeeds. The input files used to generate configure are not patched, because doing so triggers some packages to regenerate configure, and that step fails for some packages, breaking their build. It does not handle host packages. I'm not sure how to detect the host arch correctly, and although libraries may be being built incorrectly there at the moment, it's not causing any link failures so it seems unnecessary to fix them at the moment. Does this seem a reasonable solution? Would it be better to generalize it in some way? Could we solve some other problems at the same time? Should the code go somewhere else? Would a flag per package be better than checking every package? (It's not hard to do.) Cheers, Sam. package/pkg-autotools.mk | 17 ++++++++++ support/scripts/fix-configure-powerpc64.sh | 53 ++++++++++++++++++++++++++++++ 2 files changed, 70 insertions(+) create mode 100755 support/scripts/fix-configure-powerpc64.sh diff --git a/package/pkg-autotools.mk b/package/pkg-autotools.mk index d1cdb89..9e6a061 100644 --- a/package/pkg-autotools.mk +++ b/package/pkg-autotools.mk @@ -79,6 +79,15 @@ define LIBTOOL_PATCH_HOOK endef # +# Hook to patch common issue with configure on powerpc64{,le} failing +# to detect shared library support: +# +define CONFIGURE_FIX_POWERPC64_HOOK + @$(call MESSAGE,"Checking configure (powerpc64/powerpc64le)") + support/scripts/fix-configure-powerpc64.sh $($(PKG)_DIR) +endef + +# # Hook to gettextize the package if needed # define GETTEXTIZE_HOOK @@ -255,6 +264,14 @@ endif endif +# Must be added after other pre-configure hooks that might regenerate the +# configure script and overwrite the changes made here. +ifeq ($(4),target) +ifeq ($(BR2_powerpc64)$(BR2_powerpc64le),y) +$(2)_PRE_CONFIGURE_HOOKS += CONFIGURE_FIX_POWERPC64_HOOK +endif +endif + # # Build step. Only define it if not already defined by the package .mk # file. diff --git a/support/scripts/fix-configure-powerpc64.sh b/support/scripts/fix-configure-powerpc64.sh new file mode 100755 index 0000000..e4d676e --- /dev/null +++ b/support/scripts/fix-configure-powerpc64.sh @@ -0,0 +1,53 @@ +#!/bin/bash + +# This is a script to find, and correct, a problem with old versions of +# configure that affect powerpc64 and powerpc64le. + +# The issue causes configure to incorrectly determine that shared library +# support is not present in the linker. This causes the package to build a +# static library rather than a dynamic one and although the build will succeed, +# it may cause packages that link with the static library it to fail due to +# undefined symbols. + +# This script searches for files named 'configure' that appear to have this +# issue (by searching for a known bad pattern) and patching them. + +set -e + +if [ $# -ne 1 ]; then + echo "Usage: $0 " + exit 2 +fi + +pkg="$1" +files=$(cd "$pkg" && find . -name configure -exec grep -qF 'Generated by GNU Autoconf' {} \; -exec grep -qF 'ppc*-*linux*|powerpc*-*linux*)' {} \; -print) +suffix=.powerpc64.orig + +# --ignore-whitespace is needed because some packages have included +# copies of configure scripts where tabs have been replaced with spaces. +for c in $files; do + patch --ignore-whitespace --suffix=$suffix "$pkg"/"$c" <<'EOF' +*** a/configure 2016-11-07 14:04:47.444117880 +1100 +--- b/configure 2016-11-07 14:05:03.652181547 +1100 +*************** +*** 1302,1308 **** + x86_64-*linux*) + LD="${LD-ld} -m elf_x86_64" + ;; +! ppc*-*linux*|powerpc*-*linux*) + LD="${LD-ld} -m elf64ppc" + ;; + s390*-*linux*|s390*-*tpf*) +--- 1302,1311 ---- + x86_64-*linux*) + LD="${LD-ld} -m elf_x86_64" + ;; +! powerpcle-*linux*) +! LD="${LD-ld} -m elf64lppc" +! ;; +! powerpc-*linux*) + LD="${LD-ld} -m elf64ppc" + ;; + s390*-*linux*|s390*-*tpf*) +EOF +done