| Message ID | 1484112575-13194-3-git-send-email-peterx@redhat.com |
|---|---|
| State | New |
| Headers | show |
On Wed, Jan 11, 2017 at 01:29:35PM +0800, Peter Xu wrote: > run_task.sh is getting slow. This patch is trying to make it faster by > running the tests concurrently. > > We provide a new parameter "-j" for the run_tests.sh, which can be used > to specify how many run queues we want for the tests. Default queue > length is 1, which is the old behavior. > > Quick test on my laptop (4 cores, 2 threads each) shows 3x speed boost: > > |-----------------+-----------| > | command | time used | > |-----------------+-----------| > | run_test.sh | 75s | > | run_test.sh -j8 | 27s | > |-----------------+-----------| > > Signed-off-by: Peter Xu <peterx@redhat.com> > --- > run_tests.sh | 12 ++++++++++-- > scripts/common.bash | 16 +++++++++++++++- > 2 files changed, 25 insertions(+), 3 deletions(-) > > diff --git a/run_tests.sh b/run_tests.sh > index b6a1059..477d4fb 100755 > --- a/run_tests.sh > +++ b/run_tests.sh > @@ -13,10 +13,11 @@ function usage() > { > cat <<EOF > > -Usage: $0 [-g group] [-h] [-v] > +Usage: $0 [-g group] [-h] [-v] [-j num_run_queues] > > -g: Only execute tests in the given group > -h: Output this help text > + -j: Execute tests in parallel > -v: Enables verbose mode > > Set the environment variable QEMU=/path/to/qemu-system-ARCH to > @@ -28,7 +29,7 @@ EOF > RUNTIME_arch_run="./$TEST_DIR/run" > source scripts/runtime.bash > > -while getopts "g:hv" opt; do > +while getopts "g:hj:v" opt; do > case $opt in > g) > only_group=$OPTARG > @@ -37,6 +38,13 @@ while getopts "g:hv" opt; do > usage > exit > ;; > + j) > + unittest_run_queues=$OPTARG > + if (( $unittest_run_queues <= 0 )); then > + echo "Invalid -j option: $unittest_run_queues" > + exit 1 We should probably use 'exit 2' here, and below in the *) case. > + fi > + ;; > v) > verbose="yes" > ;; > diff --git a/scripts/common.bash b/scripts/common.bash > index 2dd7360..ef103ee 100644 > --- a/scripts/common.bash > +++ b/scripts/common.bash > @@ -1,11 +1,19 @@ > : ${unittest_log_dir:=logs} > +: ${unittest_run_queues:=1} > > function run_task() > { > local testname="$2" > > + while (( $(jobs | wc -l) == $unittest_run_queues )); do > + # wait for any background test to finish > + wait -n > + done > + > RUNTIME_log_file="${unittest_log_dir}/${testname}.log" > - "$@" > + > + # start the testcase in the background > + "$@" & If you check the logs before and after applying this patch series you'll see a bunch of "stty: 'standard input': Inappropriate ioctl for device" are now present. These messages come from the stty calls in run_qemu, which we need to avoid the loss of terminal echo when QEMU aborts. We can get rid of these new "inappropriate ioctl" messages by changing the above line to "$@" <$(tty) & > } > > function for_each_unittest() > @@ -22,6 +30,8 @@ function for_each_unittest() > local accel > local timeout > > + trap "wait; exit 130" SIGINT > + > exec {fd}<"$unittests" > > while read -u $fd line; do > @@ -55,5 +65,9 @@ function for_each_unittest() > fi > done > run_task "$cmd" "$testname" "$groups" "$smp" "$kernel" "$opts" "$arch" "$check" "$accel" "$timeout" > + > + # wait until all task finish Still not quite right :-) s/task/tasks/ > + wait > + > exec {fd}<&- > } > -- > 2.7.4 > > Thanks, drew
On Wed, Jan 11, 2017 at 12:00:23PM +0100, Andrew Jones wrote: > On Wed, Jan 11, 2017 at 01:29:35PM +0800, Peter Xu wrote: > > run_task.sh is getting slow. This patch is trying to make it faster by > > running the tests concurrently. > > > > We provide a new parameter "-j" for the run_tests.sh, which can be used > > to specify how many run queues we want for the tests. Default queue > > length is 1, which is the old behavior. > > > > Quick test on my laptop (4 cores, 2 threads each) shows 3x speed boost: > > > > |-----------------+-----------| > > | command | time used | > > |-----------------+-----------| > > | run_test.sh | 75s | > > | run_test.sh -j8 | 27s | > > |-----------------+-----------| > > > > Signed-off-by: Peter Xu <peterx@redhat.com> > > --- > > run_tests.sh | 12 ++++++++++-- > > scripts/common.bash | 16 +++++++++++++++- > > 2 files changed, 25 insertions(+), 3 deletions(-) > > > > diff --git a/run_tests.sh b/run_tests.sh > > index b6a1059..477d4fb 100755 > > --- a/run_tests.sh > > +++ b/run_tests.sh > > @@ -13,10 +13,11 @@ function usage() > > { > > cat <<EOF > > > > -Usage: $0 [-g group] [-h] [-v] > > +Usage: $0 [-g group] [-h] [-v] [-j num_run_queues] > > > > -g: Only execute tests in the given group > > -h: Output this help text > > + -j: Execute tests in parallel > > -v: Enables verbose mode > > > > Set the environment variable QEMU=/path/to/qemu-system-ARCH to > > @@ -28,7 +29,7 @@ EOF > > RUNTIME_arch_run="./$TEST_DIR/run" > > source scripts/runtime.bash > > > > -while getopts "g:hv" opt; do > > +while getopts "g:hj:v" opt; do > > case $opt in > > g) > > only_group=$OPTARG > > @@ -37,6 +38,13 @@ while getopts "g:hv" opt; do > > usage > > exit > > ;; > > + j) > > + unittest_run_queues=$OPTARG > > + if (( $unittest_run_queues <= 0 )); then > > + echo "Invalid -j option: $unittest_run_queues" > > + exit 1 > > We should probably use 'exit 2' here, and below in the *) case. > > > + fi > > + ;; > > v) > > verbose="yes" > > ;; > > diff --git a/scripts/common.bash b/scripts/common.bash > > index 2dd7360..ef103ee 100644 > > --- a/scripts/common.bash > > +++ b/scripts/common.bash > > @@ -1,11 +1,19 @@ > > : ${unittest_log_dir:=logs} > > +: ${unittest_run_queues:=1} > > > > function run_task() > > { > > local testname="$2" > > > > + while (( $(jobs | wc -l) == $unittest_run_queues )); do > > + # wait for any background test to finish > > + wait -n > > + done > > + > > RUNTIME_log_file="${unittest_log_dir}/${testname}.log" > > - "$@" > > + > > + # start the testcase in the background > > + "$@" & > > If you check the logs before and after applying this patch series you'll > see a bunch of "stty: 'standard input': Inappropriate ioctl for device" > are now present. These messages come from the stty calls in run_qemu, > which we need to avoid the loss of terminal echo when QEMU aborts. We > can get rid of these new "inappropriate ioctl" messages by changing the > above line to > > "$@" <$(tty) & Actually I just came up with a better solution for this. Since we don't need input to our unit tests, i.e. we never attempt to read from the serial port within them, then we can just use /dev/null for stdin. That allows us to leave your code above alone, and also remove the stty stuff in run_qemu. I'll send a patch. drew > > > > } > > > > function for_each_unittest() > > @@ -22,6 +30,8 @@ function for_each_unittest() > > local accel > > local timeout > > > > + trap "wait; exit 130" SIGINT > > + > > exec {fd}<"$unittests" > > > > while read -u $fd line; do > > @@ -55,5 +65,9 @@ function for_each_unittest() > > fi > > done > > run_task "$cmd" "$testname" "$groups" "$smp" "$kernel" "$opts" "$arch" "$check" "$accel" "$timeout" > > + > > + # wait until all task finish > > Still not quite right :-) s/task/tasks/ > > > + wait > > + > > exec {fd}<&- > > } > > -- > > 2.7.4 > > > > > > Thanks, > drew >
On Wed, Jan 11, 2017 at 02:09:34PM +0100, Andrew Jones wrote: > On Wed, Jan 11, 2017 at 12:00:23PM +0100, Andrew Jones wrote: > > On Wed, Jan 11, 2017 at 01:29:35PM +0800, Peter Xu wrote: > > > run_task.sh is getting slow. This patch is trying to make it faster by > > > running the tests concurrently. > > > > > > We provide a new parameter "-j" for the run_tests.sh, which can be used > > > to specify how many run queues we want for the tests. Default queue > > > length is 1, which is the old behavior. > > > > > > Quick test on my laptop (4 cores, 2 threads each) shows 3x speed boost: > > > > > > |-----------------+-----------| > > > | command | time used | > > > |-----------------+-----------| > > > | run_test.sh | 75s | > > > | run_test.sh -j8 | 27s | > > > |-----------------+-----------| > > > > > > Signed-off-by: Peter Xu <peterx@redhat.com> > > > --- > > > run_tests.sh | 12 ++++++++++-- > > > scripts/common.bash | 16 +++++++++++++++- > > > 2 files changed, 25 insertions(+), 3 deletions(-) > > > > > > diff --git a/run_tests.sh b/run_tests.sh > > > index b6a1059..477d4fb 100755 > > > --- a/run_tests.sh > > > +++ b/run_tests.sh > > > @@ -13,10 +13,11 @@ function usage() > > > { > > > cat <<EOF > > > > > > -Usage: $0 [-g group] [-h] [-v] > > > +Usage: $0 [-g group] [-h] [-v] [-j num_run_queues] > > > > > > -g: Only execute tests in the given group > > > -h: Output this help text > > > + -j: Execute tests in parallel > > > -v: Enables verbose mode > > > > > > Set the environment variable QEMU=/path/to/qemu-system-ARCH to > > > @@ -28,7 +29,7 @@ EOF > > > RUNTIME_arch_run="./$TEST_DIR/run" > > > source scripts/runtime.bash > > > > > > -while getopts "g:hv" opt; do > > > +while getopts "g:hj:v" opt; do > > > case $opt in > > > g) > > > only_group=$OPTARG > > > @@ -37,6 +38,13 @@ while getopts "g:hv" opt; do > > > usage > > > exit > > > ;; > > > + j) > > > + unittest_run_queues=$OPTARG > > > + if (( $unittest_run_queues <= 0 )); then > > > + echo "Invalid -j option: $unittest_run_queues" > > > + exit 1 > > > > We should probably use 'exit 2' here, and below in the *) case. Will fix this one first, with another one line patch for the below one (we can either take that new one, or squash it into this patch). > > > > > + fi > > > + ;; > > > v) > > > verbose="yes" > > > ;; > > > diff --git a/scripts/common.bash b/scripts/common.bash > > > index 2dd7360..ef103ee 100644 > > > --- a/scripts/common.bash > > > +++ b/scripts/common.bash > > > @@ -1,11 +1,19 @@ > > > : ${unittest_log_dir:=logs} > > > +: ${unittest_run_queues:=1} > > > > > > function run_task() > > > { > > > local testname="$2" > > > > > > + while (( $(jobs | wc -l) == $unittest_run_queues )); do > > > + # wait for any background test to finish > > > + wait -n > > > + done > > > + > > > RUNTIME_log_file="${unittest_log_dir}/${testname}.log" > > > - "$@" > > > + > > > + # start the testcase in the background > > > + "$@" & > > > > If you check the logs before and after applying this patch series you'll > > see a bunch of "stty: 'standard input': Inappropriate ioctl for device" > > are now present. These messages come from the stty calls in run_qemu, > > which we need to avoid the loss of terminal echo when QEMU aborts. We > > can get rid of these new "inappropriate ioctl" messages by changing the > > above line to > > > > "$@" <$(tty) & > > Actually I just came up with a better solution for this. Since we don't > need input to our unit tests, i.e. we never attempt to read from the > serial port within them, then we can just use /dev/null for stdin. That > allows us to leave your code above alone, and also remove the stty stuff > in run_qemu. I'll send a patch. Thanks, then I'll keep it as it is. > > drew > > > > > > > > } > > > > > > function for_each_unittest() > > > @@ -22,6 +30,8 @@ function for_each_unittest() > > > local accel > > > local timeout > > > > > > + trap "wait; exit 130" SIGINT > > > + > > > exec {fd}<"$unittests" > > > > > > while read -u $fd line; do > > > @@ -55,5 +65,9 @@ function for_each_unittest() > > > fi > > > done > > > run_task "$cmd" "$testname" "$groups" "$smp" "$kernel" "$opts" "$arch" "$check" "$accel" "$timeout" > > > + > > > + # wait until all task finish > > > > Still not quite right :-) s/task/tasks/ Sorry! Fixing up. -- peterx
diff --git a/run_tests.sh b/run_tests.sh index b6a1059..477d4fb 100755 --- a/run_tests.sh +++ b/run_tests.sh @@ -13,10 +13,11 @@ function usage() { cat <<EOF -Usage: $0 [-g group] [-h] [-v] +Usage: $0 [-g group] [-h] [-v] [-j num_run_queues] -g: Only execute tests in the given group -h: Output this help text + -j: Execute tests in parallel -v: Enables verbose mode Set the environment variable QEMU=/path/to/qemu-system-ARCH to @@ -28,7 +29,7 @@ EOF RUNTIME_arch_run="./$TEST_DIR/run" source scripts/runtime.bash -while getopts "g:hv" opt; do +while getopts "g:hj:v" opt; do case $opt in g) only_group=$OPTARG @@ -37,6 +38,13 @@ while getopts "g:hv" opt; do usage exit ;; + j) + unittest_run_queues=$OPTARG + if (( $unittest_run_queues <= 0 )); then + echo "Invalid -j option: $unittest_run_queues" + exit 1 + fi + ;; v) verbose="yes" ;; diff --git a/scripts/common.bash b/scripts/common.bash index 2dd7360..ef103ee 100644 --- a/scripts/common.bash +++ b/scripts/common.bash @@ -1,11 +1,19 @@ : ${unittest_log_dir:=logs} +: ${unittest_run_queues:=1} function run_task() { local testname="$2" + while (( $(jobs | wc -l) == $unittest_run_queues )); do + # wait for any background test to finish + wait -n + done + RUNTIME_log_file="${unittest_log_dir}/${testname}.log" - "$@" + + # start the testcase in the background + "$@" & } function for_each_unittest() @@ -22,6 +30,8 @@ function for_each_unittest() local accel local timeout + trap "wait; exit 130" SIGINT + exec {fd}<"$unittests" while read -u $fd line; do @@ -55,5 +65,9 @@ function for_each_unittest() fi done run_task "$cmd" "$testname" "$groups" "$smp" "$kernel" "$opts" "$arch" "$check" "$accel" "$timeout" + + # wait until all task finish + wait + exec {fd}<&- }
run_task.sh is getting slow. This patch is trying to make it faster by running the tests concurrently. We provide a new parameter "-j" for the run_tests.sh, which can be used to specify how many run queues we want for the tests. Default queue length is 1, which is the old behavior. Quick test on my laptop (4 cores, 2 threads each) shows 3x speed boost: |-----------------+-----------| | command | time used | |-----------------+-----------| | run_test.sh | 75s | | run_test.sh -j8 | 27s | |-----------------+-----------| Signed-off-by: Peter Xu <peterx@redhat.com> --- run_tests.sh | 12 ++++++++++-- scripts/common.bash | 16 +++++++++++++++- 2 files changed, 25 insertions(+), 3 deletions(-)