From patchwork Mon Jan 9 04:04:54 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 712498 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3txhR82pzdz9t0P for ; Mon, 9 Jan 2017 15:06:12 +1100 (AEDT) Received: from localhost ([::1]:37041 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cQRDm-0006l4-1X for incoming@patchwork.ozlabs.org; Sun, 08 Jan 2017 23:06:10 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57541) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cQRCn-00067S-Ib for qemu-devel@nongnu.org; Sun, 08 Jan 2017 23:05:12 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cQRCm-0002XW-Bl for qemu-devel@nongnu.org; Sun, 08 Jan 2017 23:05:09 -0500 Received: from mx1.redhat.com ([209.132.183.28]:34636) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cQRCm-0002X9-4a for qemu-devel@nongnu.org; Sun, 08 Jan 2017 23:05:08 -0500 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5B902811D8; Mon, 9 Jan 2017 04:05:08 +0000 (UTC) Received: from pxdev.xzpeter.org (vpn1-4-248.pek2.redhat.com [10.72.4.248]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id v0944ufj029859; Sun, 8 Jan 2017 23:05:05 -0500 From: Peter Xu To: qemu-devel@nongnu.org, kvm@vger.kernel.org Date: Mon, 9 Jan 2017 12:04:54 +0800 Message-Id: <1483934694-32425-3-git-send-email-peterx@redhat.com> In-Reply-To: <1483934694-32425-1-git-send-email-peterx@redhat.com> References: <1483934694-32425-1-git-send-email-peterx@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Mon, 09 Jan 2017 04:05:08 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [kvm-unit-tests PATCH v4 2/2] run_tests: allow run tests in parallel X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , Andrew Jones , peterx@redhat.com, =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?= Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" run_task.sh is getting slow. This patch is trying to make it faster by running the tests concurrently. We provide a new parameter "-j" for the run_tests.sh, which can be used to specify how many run queues we want for the tests. Default queue length is 1, which is the old behavior. Quick test on my laptop (4 cores, 2 threads each) shows 3x speed boost: |-----------------+-----------| | command | time used | |-----------------+-----------| | run_test.sh | 75s | | run_test.sh -j8 | 27s | |-----------------+-----------| Signed-off-by: Peter Xu --- run_tests.sh | 12 ++++++++++-- scripts/common.bash | 16 +++++++++++++++- 2 files changed, 25 insertions(+), 3 deletions(-) diff --git a/run_tests.sh b/run_tests.sh index 1e36d66..795cf73 100755 --- a/run_tests.sh +++ b/run_tests.sh @@ -13,10 +13,11 @@ function usage() { cat < 0 )); then + echo "Invalid -j option: $unittest_run_queues" + exit 1 + fi + ;; v) verbose="yes" ;; diff --git a/scripts/common.bash b/scripts/common.bash index 2dd7360..83aebf8 100644 --- a/scripts/common.bash +++ b/scripts/common.bash @@ -1,11 +1,19 @@ : ${unittest_log_dir:=logs} +: ${unittest_run_queues:=1} function run_task() { local testname="$2" + while (( "$(jobs | wc -l)" == $unittest_run_queues )); do + # wait for any background test to finish + wait -n + done + RUNTIME_log_file="${unittest_log_dir}/${testname}.log" - "$@" + + # start the testcase in the background + "$@" & } function for_each_unittest() @@ -22,6 +30,8 @@ function for_each_unittest() local accel local timeout + trap "wait; exit 130" SIGINT + exec {fd}<"$unittests" while read -u $fd line; do @@ -55,5 +65,9 @@ function for_each_unittest() fi done run_task "$cmd" "$testname" "$groups" "$smp" "$kernel" "$opts" "$arch" "$check" "$accel" "$timeout" + + # wait all task finish + wait + exec {fd}<&- }