| Message ID | 20230302184606.418541-6-berrange@redhat.com |
|---|---|
| State | New |
| Headers | show |
| Series | iotests: make meson aware of individual I/O tests | expand |
On 02/03/2023 19.46, Daniel P. Berrangé wrote: > Currently meson registers a single test that invokes an entire group of > I/O tests, hiding the test granularity from meson. There are various > downsides of doing this > > * You cannot ask 'meson test' to invoke a single I/O test > * The meson test timeout can't be applied to the individual > tests > * Meson only gets a pass/fail for the overall I/O test group > not individual tests > * If a CI job gets killed by the GitLab timeout, we don't > get visibility into how far through the I/O tests > execution got. > > This switches meson to perform test discovery by invoking 'check' in > dry-run mode. It then registers one meson test case for each I/O > test. Parallel execution remains disabled since the I/O tests do not > use self contained execution environments and thus conflict with > each other. > > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> > --- > tests/qemu-iotests/meson.build | 33 +++++++++++++++++++++++++++------ > 1 file changed, 27 insertions(+), 6 deletions(-) > > diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build > index 323a4acb6a..48c82085af 100644 > --- a/tests/qemu-iotests/meson.build > +++ b/tests/qemu-iotests/meson.build > @@ -32,16 +32,37 @@ foreach k, v : emulators > endif > endforeach > > +qemu_iotests_check_cmd = files('check') > + > foreach format, speed: qemu_iotests_formats > if speed == 'quick' > suites = 'block' > else > suites = ['block-' + speed, speed] > endif > - test('qemu-iotests ' + format, sh, args: [files('../check-block.sh'), format], > - depends: qemu_iotests_binaries, env: qemu_iotests_env, > - protocol: 'tap', > - suite: suites, > - timeout: 0, > - is_parallel: false) > + > + args = ['-tap', '-' + format] > + if speed == 'quick' > + args += ['-g', 'auto'] > + endif > + > + rc = run_command( > + [qemu_iotests_check_cmd] + args + ['-n'], > + check: true, > + ) > + > + foreach item: rc.stdout().strip().split() > + message('Adding test qemu-iotests-' + format + '-' + item) This message spoils the output during "configure" quite a bit, please remove that line. Apart from that, patch looks fine to me! Thomas > + args = ['-tap', '-' + format, item, > + '--source-dir', meson.current_source_dir(), > + '--build-dir', meson.current_build_dir()] > + test('qemu-iotests-' + format + '-' + item, > + qemu_iotests_check_cmd, > + args: args, > + is_parallel: false, > + depends: qemu_iotests_binaries, > + env: qemu_iotests_env, > + protocol: 'tap', > + suite: suites) > + endforeach > endforeach
On Fri, Mar 03, 2023 at 10:34:11AM +0100, Thomas Huth wrote: > On 02/03/2023 19.46, Daniel P. Berrangé wrote: > > Currently meson registers a single test that invokes an entire group of > > I/O tests, hiding the test granularity from meson. There are various > > downsides of doing this > > > > * You cannot ask 'meson test' to invoke a single I/O test > > * The meson test timeout can't be applied to the individual > > tests > > * Meson only gets a pass/fail for the overall I/O test group > > not individual tests > > * If a CI job gets killed by the GitLab timeout, we don't > > get visibility into how far through the I/O tests > > execution got. > > > > This switches meson to perform test discovery by invoking 'check' in > > dry-run mode. It then registers one meson test case for each I/O > > test. Parallel execution remains disabled since the I/O tests do not > > use self contained execution environments and thus conflict with > > each other. > > > > Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> > > --- > > tests/qemu-iotests/meson.build | 33 +++++++++++++++++++++++++++------ > > 1 file changed, 27 insertions(+), 6 deletions(-) > > > > diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build > > index 323a4acb6a..48c82085af 100644 > > --- a/tests/qemu-iotests/meson.build > > +++ b/tests/qemu-iotests/meson.build > > @@ -32,16 +32,37 @@ foreach k, v : emulators > > endif > > endforeach > > +qemu_iotests_check_cmd = files('check') > > + > > foreach format, speed: qemu_iotests_formats > > if speed == 'quick' > > suites = 'block' > > else > > suites = ['block-' + speed, speed] > > endif > > - test('qemu-iotests ' + format, sh, args: [files('../check-block.sh'), format], > > - depends: qemu_iotests_binaries, env: qemu_iotests_env, > > - protocol: 'tap', > > - suite: suites, > > - timeout: 0, > > - is_parallel: false) > > + > > + args = ['-tap', '-' + format] > > + if speed == 'quick' > > + args += ['-g', 'auto'] > > + endif > > + > > + rc = run_command( > > + [qemu_iotests_check_cmd] + args + ['-n'], > > + check: true, > > + ) > > + > > + foreach item: rc.stdout().strip().split() > > + message('Adding test qemu-iotests-' + format + '-' + item) > > This message spoils the output during "configure" quite a bit, please remove > that line. Yes, that wasn't supposed to be left it, it was me debugging :-) > > Apart from that, patch looks fine to me! > > Thomas > > > > + args = ['-tap', '-' + format, item, > > + '--source-dir', meson.current_source_dir(), > > + '--build-dir', meson.current_build_dir()] > > + test('qemu-iotests-' + format + '-' + item, > > + qemu_iotests_check_cmd, > > + args: args, > > + is_parallel: false, > > + depends: qemu_iotests_binaries, > > + env: qemu_iotests_env, > > + protocol: 'tap', > > + suite: suites) > > + endforeach > > endforeach > With regards, Daniel
diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build index 323a4acb6a..48c82085af 100644 --- a/tests/qemu-iotests/meson.build +++ b/tests/qemu-iotests/meson.build @@ -32,16 +32,37 @@ foreach k, v : emulators endif endforeach +qemu_iotests_check_cmd = files('check') + foreach format, speed: qemu_iotests_formats if speed == 'quick' suites = 'block' else suites = ['block-' + speed, speed] endif - test('qemu-iotests ' + format, sh, args: [files('../check-block.sh'), format], - depends: qemu_iotests_binaries, env: qemu_iotests_env, - protocol: 'tap', - suite: suites, - timeout: 0, - is_parallel: false) + + args = ['-tap', '-' + format] + if speed == 'quick' + args += ['-g', 'auto'] + endif + + rc = run_command( + [qemu_iotests_check_cmd] + args + ['-n'], + check: true, + ) + + foreach item: rc.stdout().strip().split() + message('Adding test qemu-iotests-' + format + '-' + item) + args = ['-tap', '-' + format, item, + '--source-dir', meson.current_source_dir(), + '--build-dir', meson.current_build_dir()] + test('qemu-iotests-' + format + '-' + item, + qemu_iotests_check_cmd, + args: args, + is_parallel: false, + depends: qemu_iotests_binaries, + env: qemu_iotests_env, + protocol: 'tap', + suite: suites) + endforeach endforeach
Currently meson registers a single test that invokes an entire group of I/O tests, hiding the test granularity from meson. There are various downsides of doing this * You cannot ask 'meson test' to invoke a single I/O test * The meson test timeout can't be applied to the individual tests * Meson only gets a pass/fail for the overall I/O test group not individual tests * If a CI job gets killed by the GitLab timeout, we don't get visibility into how far through the I/O tests execution got. This switches meson to perform test discovery by invoking 'check' in dry-run mode. It then registers one meson test case for each I/O test. Parallel execution remains disabled since the I/O tests do not use self contained execution environments and thus conflict with each other. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- tests/qemu-iotests/meson.build | 33 +++++++++++++++++++++++++++------ 1 file changed, 27 insertions(+), 6 deletions(-)