Message ID | 20210721160905.234915-1-lvivier@redhat.com |
---|---|
State | New |
Headers | show |
Series | [v4] failover: unregister ROM on unplug | expand |
On 7/21/21 6:09 PM, Laurent Vivier wrote: > The intend of failover is to allow to migrate a VM with a VFIO > networking card without disrupting the network operation by switching > to a virtio-net device during the migration. > > This simple change allows to test failover with a simulated device > like e1000e rather than a vfio device, even if it's useless in real > life it can help to debug failover. > > This is interesting to developers that want to test failover on > a system with no vfio device. Moreover it simplifies host networking > configuration as we can use the same bridge for virtio-net and > the other failover networking device. > > Without this change the migration of a system configured with failover > fails with: > > ... > -device virtio-net-pci,id=virtionet0,failover=on,... \ > -device e1000,failover_pair_id=virtionet0,... \ > ... > > (qemu) migrate ... > > Unknown ramblock "0000:00:01.1:00.0/e1000e.rom", cannot accept migration > error while loading state for instance 0x0 of device 'ram' > load of migration failed: Invalid argument > > This happens because QEMU correctly unregisters the interface vmstate but > not the ROM one. This patch fixes that. > > Signed-off-by: Laurent Vivier <lvivier@redhat.com> > --- > > Notes: > v4: > export and use pci_del_option_rom() > > v3: > remove useless space before comma > > v2: > reset has_rom to false > update commit log message > > include/hw/pci/pci.h | 2 ++ > hw/net/virtio-net.c | 1 + > hw/pci/pci.c | 3 +-- > 3 files changed, 4 insertions(+), 2 deletions(-) Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
On Wed, Jul 21, 2021 at 06:09:05PM +0200, Laurent Vivier wrote: > The intend of failover is to allow to migrate a VM with a VFIO > networking card without disrupting the network operation by switching > to a virtio-net device during the migration. > > This simple change allows to test failover with a simulated device > like e1000e rather than a vfio device, even if it's useless in real > life it can help to debug failover. > > This is interesting to developers that want to test failover on > a system with no vfio device. Moreover it simplifies host networking > configuration as we can use the same bridge for virtio-net and > the other failover networking device. > > Without this change the migration of a system configured with failover > fails with: > > ... > -device virtio-net-pci,id=virtionet0,failover=on,... \ > -device e1000,failover_pair_id=virtionet0,... \ > ... > > (qemu) migrate ... > > Unknown ramblock "0000:00:01.1:00.0/e1000e.rom", cannot accept migration > error while loading state for instance 0x0 of device 'ram' > load of migration failed: Invalid argument > > This happens because QEMU correctly unregisters the interface vmstate but > not the ROM one. This patch fixes that. > > Signed-off-by: Laurent Vivier <lvivier@redhat.com> And does failover_replug_primary roll it all back then? > --- > > Notes: > v4: > export and use pci_del_option_rom() > > v3: > remove useless space before comma > > v2: > reset has_rom to false > update commit log message > > include/hw/pci/pci.h | 2 ++ > hw/net/virtio-net.c | 1 + > hw/pci/pci.c | 3 +-- > 3 files changed, 4 insertions(+), 2 deletions(-) > > diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h > index d0f4266e3725..84707034cbf8 100644 > --- a/include/hw/pci/pci.h > +++ b/include/hw/pci/pci.h > @@ -369,6 +369,8 @@ void pci_register_vga(PCIDevice *pci_dev, MemoryRegion *mem, > void pci_unregister_vga(PCIDevice *pci_dev); > pcibus_t pci_get_bar_addr(PCIDevice *pci_dev, int region_num); > > +void pci_del_option_rom(PCIDevice *pdev); > + > int pci_add_capability(PCIDevice *pdev, uint8_t cap_id, > uint8_t offset, uint8_t size, > Error **errp); > diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c > index 16d20cdee52a..d6f03633f1b3 100644 > --- a/hw/net/virtio-net.c > +++ b/hw/net/virtio-net.c > @@ -3256,6 +3256,7 @@ static void virtio_net_handle_migration_primary(VirtIONet *n, MigrationState *s) > if (migration_in_setup(s) && !should_be_hidden) { > if (failover_unplug_primary(n, dev)) { > vmstate_unregister(VMSTATE_IF(dev), qdev_get_vmsd(dev), dev); > + pci_del_option_rom(PCI_DEVICE(dev)); > qapi_event_send_unplug_primary(dev->id); > qatomic_set(&n->failover_primary_hidden, true); > } else { > diff --git a/hw/pci/pci.c b/hw/pci/pci.c > index 23d2ae2ab232..c210d92b5ba7 100644 > --- a/hw/pci/pci.c > +++ b/hw/pci/pci.c > @@ -228,7 +228,6 @@ static PCIBus *pci_find_bus_nr(PCIBus *bus, int bus_num); > static void pci_update_mappings(PCIDevice *d); > static void pci_irq_handler(void *opaque, int irq_num, int level); > static void pci_add_option_rom(PCIDevice *pdev, bool is_default_rom, Error **); > -static void pci_del_option_rom(PCIDevice *pdev); > > static uint16_t pci_default_sub_vendor_id = PCI_SUBVENDOR_ID_REDHAT_QUMRANET; > static uint16_t pci_default_sub_device_id = PCI_SUBDEVICE_ID_QEMU; > @@ -2429,7 +2428,7 @@ static void pci_add_option_rom(PCIDevice *pdev, bool is_default_rom, > pci_register_bar(pdev, PCI_ROM_SLOT, 0, &pdev->rom); > } > > -static void pci_del_option_rom(PCIDevice *pdev) > +void pci_del_option_rom(PCIDevice *pdev) > { > if (!pdev->has_rom) > return; > -- > 2.31.1
On 21/07/2021 18:19, Michael S. Tsirkin wrote: > On Wed, Jul 21, 2021 at 06:09:05PM +0200, Laurent Vivier wrote: >> The intend of failover is to allow to migrate a VM with a VFIO >> networking card without disrupting the network operation by switching >> to a virtio-net device during the migration. >> >> This simple change allows to test failover with a simulated device >> like e1000e rather than a vfio device, even if it's useless in real >> life it can help to debug failover. >> >> This is interesting to developers that want to test failover on >> a system with no vfio device. Moreover it simplifies host networking >> configuration as we can use the same bridge for virtio-net and >> the other failover networking device. >> >> Without this change the migration of a system configured with failover >> fails with: >> >> ... >> -device virtio-net-pci,id=virtionet0,failover=on,... \ >> -device e1000,failover_pair_id=virtionet0,... \ >> ... >> >> (qemu) migrate ... >> >> Unknown ramblock "0000:00:01.1:00.0/e1000e.rom", cannot accept migration >> error while loading state for instance 0x0 of device 'ram' >> load of migration failed: Invalid argument >> >> This happens because QEMU correctly unregisters the interface vmstate but >> not the ROM one. This patch fixes that. >> >> Signed-off-by: Laurent Vivier <lvivier@redhat.com> > > And does failover_replug_primary roll it all back then? It seems not. To check I have started a migration, then I have canceled it, and I have started a migration to a file (migrate "exec:cat > mig") and I have analyzed the result with the script scripts/analyze-migration.py But: * with VFIO it's not a problem as we never migrate VFIO device itself, * with an emulated PCI device, it's only a problem if we disable failover after having canceled a first try (because if we keep failover we acts like for VFIO and we unplug the card and don't migrate vmstates). This means we need a scenario like that to hit the bug: - enable failover with an emulated PCI device - migrate to another machine - cancel/abort the migration before the end of the migration - unplug the virtio-net device to disable the failover behavior, - migrate the machine again with only the emulated PCI device but as I said previously failover with emulated PCI device is only for developers and test purpose and not to use in production... Thanks, Laurent
On 21/07/2021 18:19, Michael S. Tsirkin wrote: > On Wed, Jul 21, 2021 at 06:09:05PM +0200, Laurent Vivier wrote: >> The intend of failover is to allow to migrate a VM with a VFIO >> networking card without disrupting the network operation by switching >> to a virtio-net device during the migration. >> >> This simple change allows to test failover with a simulated device >> like e1000e rather than a vfio device, even if it's useless in real >> life it can help to debug failover. >> >> This is interesting to developers that want to test failover on >> a system with no vfio device. Moreover it simplifies host networking >> configuration as we can use the same bridge for virtio-net and >> the other failover networking device. >> >> Without this change the migration of a system configured with failover >> fails with: >> >> ... >> -device virtio-net-pci,id=virtionet0,failover=on,... \ >> -device e1000,failover_pair_id=virtionet0,... \ >> ... >> >> (qemu) migrate ... >> >> Unknown ramblock "0000:00:01.1:00.0/e1000e.rom", cannot accept migration >> error while loading state for instance 0x0 of device 'ram' >> load of migration failed: Invalid argument >> >> This happens because QEMU correctly unregisters the interface vmstate but >> not the ROM one. This patch fixes that. >> >> Signed-off-by: Laurent Vivier <lvivier@redhat.com> > > And does failover_replug_primary roll it all back then? In fact, I think we cannot roll it back because we don't have all the information to do the vmstate_register() again. Perhaps we can implement a vmstate_disable()/vmstate_enable() to disable the state migration without removing the information from the list? Thanks, Laurent
On Wed, Jul 21, 2021 at 06:09:05PM +0200, Laurent Vivier wrote: > The intend of failover is to allow to migrate a VM with a VFIO > networking card without disrupting the network operation by switching > to a virtio-net device during the migration. > > This simple change allows to test failover with a simulated device > like e1000e rather than a vfio device, even if it's useless in real > life it can help to debug failover. > > This is interesting to developers that want to test failover on > a system with no vfio device. Moreover it simplifies host networking > configuration as we can use the same bridge for virtio-net and > the other failover networking device. > > Without this change the migration of a system configured with failover > fails with: > > ... > -device virtio-net-pci,id=virtionet0,failover=on,... \ > -device e1000,failover_pair_id=virtionet0,... \ > ... > > (qemu) migrate ... > > Unknown ramblock "0000:00:01.1:00.0/e1000e.rom", cannot accept migration > error while loading state for instance 0x0 of device 'ram' > load of migration failed: Invalid argument > > This happens because QEMU correctly unregisters the interface vmstate but > not the ROM one. This patch fixes that. > > Signed-off-by: Laurent Vivier <lvivier@redhat.com> Build fails on qemu-system-m68k: /usr/bin/ld: libqemu-m68k-softmmu.fa.p/hw_net_virtio-net.c.o: in function `virtio_net_handle_migration_primary': /scm/qemu/build/../hw/net/virtio-net.c:3259: undefined reference to `pci_del_option_rom' collect2: error: ld returned 1 exit status ninja: build stopped: subcommand failed. make[1]: *** [Makefile:154: run-ninja] Error 1 It's not pretty to poke at pci from generic virtio. Should we maybe wrap vmstate_unregister and pci_del_option_rom to allow removing all migrateable things related to the device in one go somehow? > --- > > Notes: > v4: > export and use pci_del_option_rom() > > v3: > remove useless space before comma > > v2: > reset has_rom to false > update commit log message > > include/hw/pci/pci.h | 2 ++ > hw/net/virtio-net.c | 1 + > hw/pci/pci.c | 3 +-- > 3 files changed, 4 insertions(+), 2 deletions(-) > > diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h > index d0f4266e3725..84707034cbf8 100644 > --- a/include/hw/pci/pci.h > +++ b/include/hw/pci/pci.h > @@ -369,6 +369,8 @@ void pci_register_vga(PCIDevice *pci_dev, MemoryRegion *mem, > void pci_unregister_vga(PCIDevice *pci_dev); > pcibus_t pci_get_bar_addr(PCIDevice *pci_dev, int region_num); > > +void pci_del_option_rom(PCIDevice *pdev); > + > int pci_add_capability(PCIDevice *pdev, uint8_t cap_id, > uint8_t offset, uint8_t size, > Error **errp); > diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c > index 16d20cdee52a..d6f03633f1b3 100644 > --- a/hw/net/virtio-net.c > +++ b/hw/net/virtio-net.c > @@ -3256,6 +3256,7 @@ static void virtio_net_handle_migration_primary(VirtIONet *n, MigrationState *s) > if (migration_in_setup(s) && !should_be_hidden) { > if (failover_unplug_primary(n, dev)) { > vmstate_unregister(VMSTATE_IF(dev), qdev_get_vmsd(dev), dev); > + pci_del_option_rom(PCI_DEVICE(dev)); > qapi_event_send_unplug_primary(dev->id); > qatomic_set(&n->failover_primary_hidden, true); > } else { > diff --git a/hw/pci/pci.c b/hw/pci/pci.c > index 23d2ae2ab232..c210d92b5ba7 100644 > --- a/hw/pci/pci.c > +++ b/hw/pci/pci.c > @@ -228,7 +228,6 @@ static PCIBus *pci_find_bus_nr(PCIBus *bus, int bus_num); > static void pci_update_mappings(PCIDevice *d); > static void pci_irq_handler(void *opaque, int irq_num, int level); > static void pci_add_option_rom(PCIDevice *pdev, bool is_default_rom, Error **); > -static void pci_del_option_rom(PCIDevice *pdev); > > static uint16_t pci_default_sub_vendor_id = PCI_SUBVENDOR_ID_REDHAT_QUMRANET; > static uint16_t pci_default_sub_device_id = PCI_SUBDEVICE_ID_QEMU; > @@ -2429,7 +2428,7 @@ static void pci_add_option_rom(PCIDevice *pdev, bool is_default_rom, > pci_register_bar(pdev, PCI_ROM_SLOT, 0, &pdev->rom); > } > > -static void pci_del_option_rom(PCIDevice *pdev) > +void pci_del_option_rom(PCIDevice *pdev) > { > if (!pdev->has_rom) > return; > -- > 2.31.1
On 03/08/2021 16:04, Michael S. Tsirkin wrote: > On Wed, Jul 21, 2021 at 06:09:05PM +0200, Laurent Vivier wrote: >> The intend of failover is to allow to migrate a VM with a VFIO >> networking card without disrupting the network operation by switching >> to a virtio-net device during the migration. >> >> This simple change allows to test failover with a simulated device >> like e1000e rather than a vfio device, even if it's useless in real >> life it can help to debug failover. >> >> This is interesting to developers that want to test failover on >> a system with no vfio device. Moreover it simplifies host networking >> configuration as we can use the same bridge for virtio-net and >> the other failover networking device. >> >> Without this change the migration of a system configured with failover >> fails with: >> >> ... >> -device virtio-net-pci,id=virtionet0,failover=on,... \ >> -device e1000,failover_pair_id=virtionet0,... \ >> ... >> >> (qemu) migrate ... >> >> Unknown ramblock "0000:00:01.1:00.0/e1000e.rom", cannot accept migration >> error while loading state for instance 0x0 of device 'ram' >> load of migration failed: Invalid argument >> >> This happens because QEMU correctly unregisters the interface vmstate but >> not the ROM one. This patch fixes that. >> >> Signed-off-by: Laurent Vivier <lvivier@redhat.com> > > > Build fails on qemu-system-m68k: > > /usr/bin/ld: libqemu-m68k-softmmu.fa.p/hw_net_virtio-net.c.o: in function `virtio_net_handle_migration_primary': > /scm/qemu/build/../hw/net/virtio-net.c:3259: undefined reference to `pci_del_option_rom' > collect2: error: ld returned 1 exit status > ninja: build stopped: subcommand failed. > make[1]: *** [Makefile:154: run-ninja] Error 1 > > It's not pretty to poke at pci from generic virtio. I agree with you. The problem is failover is implemented in virtio-net while it relies on virtio-pci hotplug capability. > Should we maybe wrap vmstate_unregister and pci_del_option_rom > to allow removing all migrateable things related to the device > in one go somehow? I'm going to have a look to see how to do. I have already a patch that moves all the failover PCI unplug/hotplug functions into the PCI device implementation (and removes all this stuff from virtio-net). Perhaps we can rely on that. The bonus with this moves is it allows to automatically unplug/hotplug any PCI device during a migration without the failover environment (but failover uses it). Thanks, Laurent > >> --- >> >> Notes: >> v4: >> export and use pci_del_option_rom() >> >> v3: >> remove useless space before comma >> >> v2: >> reset has_rom to false >> update commit log message >> >> include/hw/pci/pci.h | 2 ++ >> hw/net/virtio-net.c | 1 + >> hw/pci/pci.c | 3 +-- >> 3 files changed, 4 insertions(+), 2 deletions(-) >> >> diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h >> index d0f4266e3725..84707034cbf8 100644 >> --- a/include/hw/pci/pci.h >> +++ b/include/hw/pci/pci.h >> @@ -369,6 +369,8 @@ void pci_register_vga(PCIDevice *pci_dev, MemoryRegion *mem, >> void pci_unregister_vga(PCIDevice *pci_dev); >> pcibus_t pci_get_bar_addr(PCIDevice *pci_dev, int region_num); >> >> +void pci_del_option_rom(PCIDevice *pdev); >> + >> int pci_add_capability(PCIDevice *pdev, uint8_t cap_id, >> uint8_t offset, uint8_t size, >> Error **errp); >> diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c >> index 16d20cdee52a..d6f03633f1b3 100644 >> --- a/hw/net/virtio-net.c >> +++ b/hw/net/virtio-net.c >> @@ -3256,6 +3256,7 @@ static void virtio_net_handle_migration_primary(VirtIONet *n, MigrationState *s) >> if (migration_in_setup(s) && !should_be_hidden) { >> if (failover_unplug_primary(n, dev)) { >> vmstate_unregister(VMSTATE_IF(dev), qdev_get_vmsd(dev), dev); >> + pci_del_option_rom(PCI_DEVICE(dev)); >> qapi_event_send_unplug_primary(dev->id); >> qatomic_set(&n->failover_primary_hidden, true); >> } else { >> diff --git a/hw/pci/pci.c b/hw/pci/pci.c >> index 23d2ae2ab232..c210d92b5ba7 100644 >> --- a/hw/pci/pci.c >> +++ b/hw/pci/pci.c >> @@ -228,7 +228,6 @@ static PCIBus *pci_find_bus_nr(PCIBus *bus, int bus_num); >> static void pci_update_mappings(PCIDevice *d); >> static void pci_irq_handler(void *opaque, int irq_num, int level); >> static void pci_add_option_rom(PCIDevice *pdev, bool is_default_rom, Error **); >> -static void pci_del_option_rom(PCIDevice *pdev); >> >> static uint16_t pci_default_sub_vendor_id = PCI_SUBVENDOR_ID_REDHAT_QUMRANET; >> static uint16_t pci_default_sub_device_id = PCI_SUBDEVICE_ID_QEMU; >> @@ -2429,7 +2428,7 @@ static void pci_add_option_rom(PCIDevice *pdev, bool is_default_rom, >> pci_register_bar(pdev, PCI_ROM_SLOT, 0, &pdev->rom); >> } >> >> -static void pci_del_option_rom(PCIDevice *pdev) >> +void pci_del_option_rom(PCIDevice *pdev) >> { >> if (!pdev->has_rom) >> return; >> -- >> 2.31.1 >
On Wed, Jul 21, 2021 at 06:09:05PM +0200, Laurent Vivier wrote: > The intend of failover is to allow to migrate a VM with a VFIO > networking card without disrupting the network operation by switching > to a virtio-net device during the migration. > > This simple change allows to test failover with a simulated device > like e1000e rather than a vfio device, even if it's useless in real > life it can help to debug failover. > > This is interesting to developers that want to test failover on > a system with no vfio device. Moreover it simplifies host networking > configuration as we can use the same bridge for virtio-net and > the other failover networking device. > > Without this change the migration of a system configured with failover > fails with: > > ... > -device virtio-net-pci,id=virtionet0,failover=on,... \ > -device e1000,failover_pair_id=virtionet0,... \ > ... > > (qemu) migrate ... > > Unknown ramblock "0000:00:01.1:00.0/e1000e.rom", cannot accept migration > error while loading state for instance 0x0 of device 'ram' > load of migration failed: Invalid argument > > This happens because QEMU correctly unregisters the interface vmstate but > not the ROM one. This patch fixes that. > > Signed-off-by: Laurent Vivier <lvivier@redhat.com> I began to wonder about this. Why are we sending the option ROM at all? I think there's no need to do it for the primary ... > --- > > Notes: > v4: > export and use pci_del_option_rom() > > v3: > remove useless space before comma > > v2: > reset has_rom to false > update commit log message > > include/hw/pci/pci.h | 2 ++ > hw/net/virtio-net.c | 1 + > hw/pci/pci.c | 3 +-- > 3 files changed, 4 insertions(+), 2 deletions(-) > > diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h > index d0f4266e3725..84707034cbf8 100644 > --- a/include/hw/pci/pci.h > +++ b/include/hw/pci/pci.h > @@ -369,6 +369,8 @@ void pci_register_vga(PCIDevice *pci_dev, MemoryRegion *mem, > void pci_unregister_vga(PCIDevice *pci_dev); > pcibus_t pci_get_bar_addr(PCIDevice *pci_dev, int region_num); > > +void pci_del_option_rom(PCIDevice *pdev); > + > int pci_add_capability(PCIDevice *pdev, uint8_t cap_id, > uint8_t offset, uint8_t size, > Error **errp); > diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c > index 16d20cdee52a..d6f03633f1b3 100644 > --- a/hw/net/virtio-net.c > +++ b/hw/net/virtio-net.c > @@ -3256,6 +3256,7 @@ static void virtio_net_handle_migration_primary(VirtIONet *n, MigrationState *s) > if (migration_in_setup(s) && !should_be_hidden) { > if (failover_unplug_primary(n, dev)) { > vmstate_unregister(VMSTATE_IF(dev), qdev_get_vmsd(dev), dev); > + pci_del_option_rom(PCI_DEVICE(dev)); > qapi_event_send_unplug_primary(dev->id); > qatomic_set(&n->failover_primary_hidden, true); > } else { > diff --git a/hw/pci/pci.c b/hw/pci/pci.c > index 23d2ae2ab232..c210d92b5ba7 100644 > --- a/hw/pci/pci.c > +++ b/hw/pci/pci.c > @@ -228,7 +228,6 @@ static PCIBus *pci_find_bus_nr(PCIBus *bus, int bus_num); > static void pci_update_mappings(PCIDevice *d); > static void pci_irq_handler(void *opaque, int irq_num, int level); > static void pci_add_option_rom(PCIDevice *pdev, bool is_default_rom, Error **); > -static void pci_del_option_rom(PCIDevice *pdev); > > static uint16_t pci_default_sub_vendor_id = PCI_SUBVENDOR_ID_REDHAT_QUMRANET; > static uint16_t pci_default_sub_device_id = PCI_SUBDEVICE_ID_QEMU; > @@ -2429,7 +2428,7 @@ static void pci_add_option_rom(PCIDevice *pdev, bool is_default_rom, > pci_register_bar(pdev, PCI_ROM_SLOT, 0, &pdev->rom); > } > > -static void pci_del_option_rom(PCIDevice *pdev) > +void pci_del_option_rom(PCIDevice *pdev) > { > if (!pdev->has_rom) > return; > -- > 2.31.1
On 11/08/2021 08:50, Michael S. Tsirkin wrote: > On Wed, Jul 21, 2021 at 06:09:05PM +0200, Laurent Vivier wrote: >> The intend of failover is to allow to migrate a VM with a VFIO >> networking card without disrupting the network operation by switching >> to a virtio-net device during the migration. >> >> This simple change allows to test failover with a simulated device >> like e1000e rather than a vfio device, even if it's useless in real >> life it can help to debug failover. >> >> This is interesting to developers that want to test failover on >> a system with no vfio device. Moreover it simplifies host networking >> configuration as we can use the same bridge for virtio-net and >> the other failover networking device. >> >> Without this change the migration of a system configured with failover >> fails with: >> >> ... >> -device virtio-net-pci,id=virtionet0,failover=on,... \ >> -device e1000,failover_pair_id=virtionet0,... \ >> ... >> >> (qemu) migrate ... >> >> Unknown ramblock "0000:00:01.1:00.0/e1000e.rom", cannot accept migration >> error while loading state for instance 0x0 of device 'ram' >> load of migration failed: Invalid argument >> >> This happens because QEMU correctly unregisters the interface vmstate but >> not the ROM one. This patch fixes that. >> >> Signed-off-by: Laurent Vivier <lvivier@redhat.com> > > I began to wonder about this. Why are we sending the option ROM at all? > I think there's no need to do it for the primary ... I think there is no way to check the ROM is the same on source and destination, and it has to be. By sending the ROM: - we can check the size is the same, otherwise the migration fails, - after the migration, even if a different ROM was provided on the destination side, the guest executes the ROM provided by the source. Thanks, Laurent
diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h index d0f4266e3725..84707034cbf8 100644 --- a/include/hw/pci/pci.h +++ b/include/hw/pci/pci.h @@ -369,6 +369,8 @@ void pci_register_vga(PCIDevice *pci_dev, MemoryRegion *mem, void pci_unregister_vga(PCIDevice *pci_dev); pcibus_t pci_get_bar_addr(PCIDevice *pci_dev, int region_num); +void pci_del_option_rom(PCIDevice *pdev); + int pci_add_capability(PCIDevice *pdev, uint8_t cap_id, uint8_t offset, uint8_t size, Error **errp); diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c index 16d20cdee52a..d6f03633f1b3 100644 --- a/hw/net/virtio-net.c +++ b/hw/net/virtio-net.c @@ -3256,6 +3256,7 @@ static void virtio_net_handle_migration_primary(VirtIONet *n, MigrationState *s) if (migration_in_setup(s) && !should_be_hidden) { if (failover_unplug_primary(n, dev)) { vmstate_unregister(VMSTATE_IF(dev), qdev_get_vmsd(dev), dev); + pci_del_option_rom(PCI_DEVICE(dev)); qapi_event_send_unplug_primary(dev->id); qatomic_set(&n->failover_primary_hidden, true); } else { diff --git a/hw/pci/pci.c b/hw/pci/pci.c index 23d2ae2ab232..c210d92b5ba7 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -228,7 +228,6 @@ static PCIBus *pci_find_bus_nr(PCIBus *bus, int bus_num); static void pci_update_mappings(PCIDevice *d); static void pci_irq_handler(void *opaque, int irq_num, int level); static void pci_add_option_rom(PCIDevice *pdev, bool is_default_rom, Error **); -static void pci_del_option_rom(PCIDevice *pdev); static uint16_t pci_default_sub_vendor_id = PCI_SUBVENDOR_ID_REDHAT_QUMRANET; static uint16_t pci_default_sub_device_id = PCI_SUBDEVICE_ID_QEMU; @@ -2429,7 +2428,7 @@ static void pci_add_option_rom(PCIDevice *pdev, bool is_default_rom, pci_register_bar(pdev, PCI_ROM_SLOT, 0, &pdev->rom); } -static void pci_del_option_rom(PCIDevice *pdev) +void pci_del_option_rom(PCIDevice *pdev) { if (!pdev->has_rom) return;
The intend of failover is to allow to migrate a VM with a VFIO networking card without disrupting the network operation by switching to a virtio-net device during the migration. This simple change allows to test failover with a simulated device like e1000e rather than a vfio device, even if it's useless in real life it can help to debug failover. This is interesting to developers that want to test failover on a system with no vfio device. Moreover it simplifies host networking configuration as we can use the same bridge for virtio-net and the other failover networking device. Without this change the migration of a system configured with failover fails with: ... -device virtio-net-pci,id=virtionet0,failover=on,... \ -device e1000,failover_pair_id=virtionet0,... \ ... (qemu) migrate ... Unknown ramblock "0000:00:01.1:00.0/e1000e.rom", cannot accept migration error while loading state for instance 0x0 of device 'ram' load of migration failed: Invalid argument This happens because QEMU correctly unregisters the interface vmstate but not the ROM one. This patch fixes that. Signed-off-by: Laurent Vivier <lvivier@redhat.com> --- Notes: v4: export and use pci_del_option_rom() v3: remove useless space before comma v2: reset has_rom to false update commit log message include/hw/pci/pci.h | 2 ++ hw/net/virtio-net.c | 1 + hw/pci/pci.c | 3 +-- 3 files changed, 4 insertions(+), 2 deletions(-)