From patchwork Wed Jul 1 08:10:46 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fabio Porcedda X-Patchwork-Id: 489897 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by ozlabs.org (Postfix) with ESMTP id 8EDDF1402CD for ; Wed, 1 Jul 2015 18:11:06 +1000 (AEST) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b=ys3h4Ias; dkim-atps=neutral Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id CDFEF91D07; Wed, 1 Jul 2015 08:11:05 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id A8rdjEsM6VYm; Wed, 1 Jul 2015 08:11:04 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by whitealder.osuosl.org (Postfix) with ESMTP id 5D83F91D0D; Wed, 1 Jul 2015 08:11:04 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by ash.osuosl.org (Postfix) with ESMTP id 90EBC1C21BB for ; Wed, 1 Jul 2015 08:11:03 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 4DB9433405 for ; Wed, 1 Jul 2015 08:11:03 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mq0zpML2xEKr for ; Wed, 1 Jul 2015 08:11:01 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pd0-f176.google.com (mail-pd0-f176.google.com [209.85.192.176]) by silver.osuosl.org (Postfix) with ESMTPS id A1D642DE66 for ; Wed, 1 Jul 2015 08:11:01 +0000 (UTC) Received: by pdbep18 with SMTP id ep18so21621370pdb.1 for ; Wed, 01 Jul 2015 01:11:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=tVaYNmXlyLnbc4ZA2LE64l/vG6S5VxJhJWYy2rTh7so=; b=ys3h4IasxzIC1VaWf4cw3fXMhuQGs/NJZqlkbQ7BteJaQHsZA4DiY+p1U/plyd9u9r /w9+zdPL/jMjQ/uc7MmG4ulchZ72xIlOsY5EVqAE1rwQ40hk38Yo2sLbQkA7g+uTmRdS bN3nIoqlxxeYX2zKrZ8TY19p92iQXdRVnWabVfd3M4d//djIUTiVAqzilJLPsoY85LYC jjEZjsnkFvf0JxJSDm+leotf1V9eFEqR9hTJrFZNFN7wau1HR/07V+a7L7SEJRFQNuQn /TJouFSq4RLW+UV+f1D0hWMdo7S1Bl/Q/VI9R8LUWEf3i9agbtO4jHk1m6lMGTCUX2nw 6Yiw== X-Received: by 10.66.145.74 with SMTP id ss10mr53020573pab.28.1435738261342; Wed, 01 Jul 2015 01:11:01 -0700 (PDT) Received: from ld2077.tmt.telital.com ([213.205.6.118]) by mx.google.com with ESMTPSA id kp2sm1308251pab.12.2015.07.01.01.10.57 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 01 Jul 2015 01:11:00 -0700 (PDT) From: Fabio Porcedda To: buildroot@uclibc.org Date: Wed, 1 Jul 2015 10:10:46 +0200 Message-Id: <1435738247-9015-2-git-send-email-fabio.porcedda@gmail.com> X-Mailer: git-send-email 2.4.3 In-Reply-To: <1435738247-9015-1-git-send-email-fabio.porcedda@gmail.com> References: <1435738247-9015-1-git-send-email-fabio.porcedda@gmail.com> Cc: Thomas Petazzoni Subject: [Buildroot] [PATCH v5 1/2] packages: fix and improve support for top-level parallel make X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" The boost and jack2 packages fail to build when PARALLEL_JOBS is empty so instead of using an empty PARALLEL_JOBS don't use it in the MAKE variable when top-level parallel make is being used. To simplify the use of top-level parallel make, check the MAKEFLAGS variable to know automatically if the -j option is being used, also use the "=" operator instead of the ":=" operator because the MAKEFLAGS variable can be checked only in a "recursively expanded variable". The "override" keyword must be used in order to change the automatic variable "MAKE". When the top-parallel make is being used the sub-make are called without specifying the "-j" option in order to let GNU make share the job slots specified in the top make. This is done because GNU make is able to share the job slots available between each instance of make so if you want to increase the number of jobs you just need to increase the value in the top make -j command. If we specify the -j option in each instance of make, it is less efficient, e.g. in a processor with 8 cores we specify -j9 in each instance: the number of processes goes up to 81 because each sub-make can execute 9 processes. The excessive number of processes is not a good thing because in my tests even -j16 is slower than -j9. Instead if we don't specify the -j option in the sub-make, the top make share the job slots automatically between each instance, so the number of process in this examples goes up to 9 that is faster than using up to 81 processes. e.g. when the -j3 option is specified only in the top make: possible state n. 1: process 1 - -build process 2 - -build process 3 - -build possible state n. 2: process 1 - -extract process 2 - -configure process 3 - -build possible state n. 3: process 1 - -build make -j1 process 2 - -build make -j1 process 3 - -build make -j1 Signed-off-by: Fabio Porcedda Acked-by: Arnout Vandecappelle (Essensium/Mind) --- Makefile | 4 ++-- package/Makefile.in | 3 ++- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/Makefile b/Makefile index 8eb21d4..b1bcf78 100644 --- a/Makefile +++ b/Makefile @@ -58,8 +58,8 @@ export HOSTARCH := $(shell uname -m | \ # # Taking into account the above considerations, if you still want to execute # this top-level Makefile in parallel comment the ".NOTPARALLEL" line and -# build using the following command: -# make BR2_JLEVEL= -j$((`getconf _NPROCESSORS_ONLN`+1)) +# use the -j option when building, e.g: +# make -j$((`getconf _NPROCESSORS_ONLN`+1)) .NOTPARALLEL: # absolute path diff --git a/package/Makefile.in b/package/Makefile.in index c02d31f..2ed7cf7 100644 --- a/package/Makefile.in +++ b/package/Makefile.in @@ -18,7 +18,8 @@ PARALLEL_JOBS := $(BR2_JLEVEL) endif MAKE1 := $(HOSTMAKE) -j1 -MAKE := $(HOSTMAKE) $(if $(PARALLEL_JOBS),-j$(PARALLEL_JOBS)) +override MAKE = $(HOSTMAKE) \ + $(if $(findstring j,$(filter-out --%,$(MAKEFLAGS))),,-j$(PARALLEL_JOBS)) ifeq ($(BR2_TOOLCHAIN_BUILDROOT),y) TARGET_VENDOR = $(call qstrip,$(BR2_TOOLCHAIN_BUILDROOT_VENDOR))