Message ID | 20231018140736.3618-1-farosas@suse.de |
---|---|
Headers | show |
Series | tests/migration-test: Allow testing older machine types | expand |
Fabiano Rosas <farosas@suse.de> writes: > This adds support for running migration-test with two different QEMU > versions to test migration compatibility. The tests automatically > choose the latest machine type supported by both QEMU versions. Let me point out two situations that came up while developing this: 1) There was a generic command line change in QEMU (-audio option) introduced this cycle that was ported to the command line used by libqtest.c. This means that when using the same command line to start both guests, the older one will not recognize the option and will break. Ideally, we would keep command lines out of the library and require each test to provide their own. Then maybe the migration code could cope with the change somehow. 2) When fixing an issue during a release cycle, that issue can still show up on the next one when using this series to test compatibility. There's probably not much we can do here, but remember the issues that were already fixed and ignore them when they appear in these compatibiliy tests. I don't see how to avoid the situations above, but I think their existence implies we cannot enable by default any sort of migration compatibility testing that uses a live guest.
On Wed, Oct 18, 2023 at 11:19:01AM -0300, Fabiano Rosas wrote: > Fabiano Rosas <farosas@suse.de> writes: > > > This adds support for running migration-test with two different QEMU > > versions to test migration compatibility. The tests automatically > > choose the latest machine type supported by both QEMU versions. > > Let me point out two situations that came up while developing this: > > 1) There was a generic command line change in QEMU (-audio option) > introduced this cycle that was ported to the command line used by > libqtest.c. This means that when using the same command line to > start both guests, the older one will not recognize the option and > will break. > > Ideally, we would keep command lines out of the library and require > each test to provide their own. Then maybe the migration code could > cope with the change somehow. Right the problem is that QEMU doesn't provide a long term stable API, but has finite sliding windows where QEMU versions can be compatible in their CLI, determined by our minimal deprecation period of 2 releases. eg you can come up with a CLI that is compatible for all of 7.1, 7.2 and 8.0 but you can't neccessarily come up with a CLI that is compatible for all of 7.1, 7.2, 8.0 and 8.1 because the lifespan of that set of versions is greater than our deprecation period. If we did't use -audio in libqtest, we could be compatible with current QEMU and some old QEMUs, but not future QEMU. If we do use -audio in libqtest, we could be compatible with current QEMU and some future QEMUs, but not old QEMUs. This is a no win scenario. Now, libvirt provides a long term stable configuration for QEMU that transcends changes in QEMU CLI. So we could come up with a live migration test between QEMU's of an arbitrary age difference, if we used libvirt to drive the process. That wouldn't really be qtest though. Could perhaps we done in the context of avocado, if we used the libvirt-python API binding, but this would be different from other avocado tests by virtue of not using the QEMU python classes for launch. > 2) When fixing an issue during a release cycle, that issue can still > show up on the next one when using this series to test > compatibility. There's probably not much we can do here, but > remember the issues that were already fixed and ignore them when > they appear in these compatibiliy tests. > > I don't see how to avoid the situations above, but I think their > existence implies we cannot enable by default any sort of migration > compatibility testing that uses a live guest. With regards, Daniel
On Wed, Oct 18, 2023 at 11:19:01AM -0300, Fabiano Rosas wrote: > Let me point out two situations that came up while developing this: > > 1) There was a generic command line change in QEMU (-audio option) > introduced this cycle that was ported to the command line used by > libqtest.c. This means that when using the same command line to > start both guests, the older one will not recognize the option and > will break. > > Ideally, we would keep command lines out of the library and require > each test to provide their own. Then maybe the migration code could > cope with the change somehow. Yeah this is unfortunate, and I hit it as soon as trying this out with an old binary. One other thing on top of this: the cross binary tests for different devices, where we used to hit issues when some specific pci device or a pcie root port failed to be compatible with old binaries, iirc. I'm still not sure what's the best way to do that, though, especially if the migration failure can only trigger when a device driver setup the device first. Maybe we can start with migrating a device with the reset status between binaries. Having this series is definitely better than nothing already. Thanks,