From patchwork Fri Sep 30 13:25:31 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Marc-Andr=C3=A9_Lureau?= X-Patchwork-Id: 677039 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3slsfW4BJ7z9s3s for ; Fri, 30 Sep 2016 23:26:43 +1000 (AEST) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b=FsIIg98I; dkim-atps=neutral Received: from localhost ([::1]:44533 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bpxpm-0004XI-3A for incoming@patchwork.ozlabs.org; Fri, 30 Sep 2016 09:26:38 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58423) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bpxoy-00046U-3N for qemu-devel@nongnu.org; Fri, 30 Sep 2016 09:25:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bpxov-0006mU-B3 for qemu-devel@nongnu.org; Fri, 30 Sep 2016 09:25:47 -0400 Received: from mail-lf0-x22d.google.com ([2a00:1450:4010:c07::22d]:36143) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bpxou-0006ly-SA for qemu-devel@nongnu.org; Fri, 30 Sep 2016 09:25:45 -0400 Received: by mail-lf0-x22d.google.com with SMTP id g62so105772088lfe.3 for ; Fri, 30 Sep 2016 06:25:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=BIuTWPnOvTyom2hwjHIUdLn0VfzjRwFwVuC/5sUj5Mw=; b=FsIIg98Ib5ZpoMJIg/Vv1WGqpnlDOeK0UsCIO/CQuaFhwGQBQoc2j/WSkI/Q0ir/xN MlJJJFSk+6IyGD7n0GW6LMsbHXMlKSf6DsoBXsbEoEDRmvZ08TpkEhA7IhJSs06j3Ly1 l9CH/kfJ0B8SLDcxmq3DBiIAG0PfNHInMfTQuwwj3v0x8XaP54StkrErRhyIC2VjfQ00 MWZGrC2OKvUoNYk7NTRT/xbUVFDMroXyuXOdmscQWXyG141Z5Y59gMw2/Ve+t1nI3C8y nva6WWzMcwx7vZtLMjiyUqhsA7NW29nvSVQtBUVpVFo7NPLDMyt1t4wJWO8vm92klLZF hu9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=BIuTWPnOvTyom2hwjHIUdLn0VfzjRwFwVuC/5sUj5Mw=; b=Rxc9NCCsV5/RvlOkYeIqUUbwCGkMQWyqCK7JjHtr4rnR0rgkADOYHzdK5ic6k0C6ou Nf1PgXvEhyXhd5rtusChFxmWgDWOeNZqkgs1vOY2tPxr1x3958O5OpD5aidgMSl6oak6 0IPCrt1S9wsRDHQC4RlpBrEWtwlU1egxLno/UtGs1OXscG6Mjc+pfUyiyEI9Dru0CV38 8B6TaEIpm64wp2BDMQ1iLQnRCSpF6l12uHp0RxjbAx2uzAuB+oQ5oTTWaKfqTeNMeV8F gQ/wtYuheq+PVrX1hYqQbeBsBlQCqy2S+nIcbIe9gbFyao6m6phNv6CwVI6wUDmd6xt3 Wesg== X-Gm-Message-State: AA6/9Rk5bp2KmJfHvv6wmZlHOdQipumyKZECpoMiBFKeVHtPamDJ8TxK/o92c6m+DbbJ1rsl9p0AFM29GVOg8g== X-Received: by 10.25.15.229 with SMTP id 98mr2547911lfp.24.1475241942853; Fri, 30 Sep 2016 06:25:42 -0700 (PDT) MIME-Version: 1.0 References: <1475136691-920-1-git-send-email-famz@redhat.com> In-Reply-To: <1475136691-920-1-git-send-email-famz@redhat.com> From: =?UTF-8?B?TWFyYy1BbmRyw6kgTHVyZWF1?= Date: Fri, 30 Sep 2016 13:25:31 +0000 Message-ID: To: Fam Zheng , qemu-devel@nongnu.org X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:4010:c07::22d X-Content-Filtered-By: Mailman/MimeDel 2.1.21 Subject: Re: [Qemu-devel] [PATCH v2] tests: Run qtest cases in parallel X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, jsnow@redhat.com Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" Hi On Thu, Sep 29, 2016 at 12:13 PM Fam Zheng wrote: > Previously all qtest cases in a category, such as check-qtest-y, are > executed in a single long gtester command. This patch separates each > test into its own make target to allow better parallism, unless gcov is > configured. > > Slightly reorganize the gcov enabled case too, but the execution > sequence is kept. > > This saves >50% of the time (see below), which means a lot for patchew > automatic testing given how 'make check' takes part in testings. > > On my machine: > > before: > > $ make docker-test-quick@fedora J=8 &>/dev/null && \ > time make docker-test-quick@fedora J=8 >/dev/null; > > real 1m47.090s > user 0m1.599s > sys 0m0.258s > > after: > > real 0m54.067s > user 0m1.447s > sys 0m0.253s > > Signed-off-by: Fam Zheng > > --- > > v2: Fix gcov case by not changing it. [Daniel] > --- > tests/Makefile.include | 48 > ++++++++++++++++++++++++++++++++++++------------ > 1 file changed, 36 insertions(+), 12 deletions(-) > > diff --git a/tests/Makefile.include b/tests/Makefile.include > index 8162f6f..575030a 100644 > --- a/tests/Makefile.include > +++ b/tests/Makefile.include > @@ -729,27 +729,51 @@ GCOV_OPTIONS = -n $(if $(V),-f,) > # gtester tests, possibly with verbose output > > .PHONY: $(patsubst %, check-qtest-%, $(QTEST_TARGETS)) > -$(patsubst %, check-qtest-%, $(QTEST_TARGETS)): check-qtest-%: > $(check-qtest-y) > - $(if $(CONFIG_GCOV),@rm -f *.gcda */*.gcda */*/*.gcda > */*/*/*.gcda,) > - $(call quiet-command,QTEST_QEMU_BINARY=$*-softmmu/qemu-system-$* \ > + > +run-tests = $(call quiet-command,\ > + $(if $(QTEST_TARGET), \ > + > QTEST_QEMU_BINARY=$(QTEST_TARGET)-softmmu/qemu-system-$(QTEST_TARGET)) \ > QTEST_QEMU_IMG=qemu-img$(EXESUF) \ > MALLOC_PERTURB_=$${MALLOC_PERTURB_:-$$((RANDOM % 255 + > 1))} \ > - gtester $(GTESTER_OPTIONS) -m=$(SPEED) $(check-qtest-$*-y) > $(check-qtest-generic-y),"GTESTER $@") > - $(if $(CONFIG_GCOV),@for f in $(gcov-files-$*-y) > $(gcov-files-generic-y); do \ > + gtester $1 $(GTESTER_OPTIONS) -m=$(SPEED),"GTESTER $2") > + > +ifneq ($(CONFIG_GCOV),) > + > +check-qtest-%: QTEST_TARGET=$* > +$(patsubst %, check-qtest-%, $(QTEST_TARGETS)): check-qtest-%: > $(check-qtest-y) > + @rm -f *.gcda */*.gcda */*/*.gcda */*/*/*.gcda > + $(call run-tests, $(check-qtest-$*-y), $@) > + @for f in $(gcov-files-$*-y) $(gcov-files-generic-y); do \ > echo Gcov report for $$f:;\ > $(GCOV) $(GCOV_OPTIONS) $$f -o `dirname $$f`; \ > - done,) > + done > > .PHONY: $(patsubst %, check-%, $(check-unit-y)) > $(patsubst %, check-%, $(check-unit-y)): check-%: % > - $(if $(CONFIG_GCOV),@rm -f *.gcda */*.gcda */*/*.gcda > */*/*/*.gcda,) > - $(call quiet-command, \ > - MALLOC_PERTURB_=$${MALLOC_PERTURB_:-$$((RANDOM % 255 + > 1))} \ > - gtester $(GTESTER_OPTIONS) -m=$(SPEED) $*,"GTESTER $*") > - $(if $(CONFIG_GCOV),@for f in $(gcov-files-$(subst tests/,,$*)-y) > $(gcov-files-generic-y); do \ > + @rm -f *.gcda */*.gcda */*/*.gcda */*/*/*.gcda > + $(call run-tests, $*, $*) > + @for f in $(gcov-files-$(subst tests/,,$*)-y) > $(gcov-files-generic-y); do \ > echo Gcov report for $$f:;\ > $(GCOV) $(GCOV_OPTIONS) $$f -o `dirname $$f`; \ > - done,) > + done > + > +else > + > +run-test-%: tests/% > + $(call run-tests, $<, $<) > + > +$(foreach target, $(QTEST_TARGETS), \ > + $(eval check-qtest-$(target): QTEST_TARGET := $(target)) \ > + $(eval check-qtest-$(target): $(patsubst tests/%, run-test-%, \ > + $(check-qtest-y) \ > + $(check-qtest-$(target)-y) \ > + $(check-qtest-generic-y))) \ > +) > +$(patsubst %, check-%, $(check-unit-y)): check-tests/%: run-test-% > + > +endif > + > +.PHONY: $(patsubst %, check-%, $(check-unit-y)) > > it's quite difficult to read, but it looks correct to me. Something I wish we had that we can perhaps add as well here (add dep on -softmmu build): $(call run-tests, $(check-qtest-$*-y), $@) @@ -767,6 +768,7 @@ run-test-%: tests/% $(foreach target, $(QTEST_TARGETS), \ $(eval check-qtest-$(target): QTEST_TARGET := $(target)) \ + $(eval check-qtest-$(target): subdir-$(target)-softmmu) \ $(eval check-qtest-$(target): $(patsubst tests/%, run-test-%, \ $(check-qtest-y) \ $(check-qtest-$(target)-y) \ I can post a seperate patch later though, Similarly, qga test: -tests/test-qga: tests/test-qga.o $(qtest-obj-y) +tests/test-qga: tests/test-qga.o $(qtest-obj-y) qemu-ga$(EXESUF) In any case Reviewed-by: Marc-André Lureau > # gtester tests with XML output > > -- > 2.7.4 > > > -- Marc-André Lureau diff --git a/tests/Makefile.include b/tests/Makefile.include index f5491c7..8e5dc72 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -743,6 +743,7 @@ run-tests = $(call quiet-command,\ ifneq ($(CONFIG_GCOV),) check-qtest-%: QTEST_TARGET=$* +check-qtest-%: subdir-$*-softmmu $(patsubst %, check-qtest-%, $(QTEST_TARGETS)): check-qtest-%: $(check-qtest-y) @rm -f *.gcda */*.gcda */*/*.gcda */*/*/*.gcda