| Message ID | 20190719043734.108462-1-aik@ozlabs.ru |
|---|---|
| State | New |
| Headers | show |
| Series | [qemu,v3] spapr_pci: Advertise BAR reallocation capability | expand |
On Fri, Jul 19, 2019 at 02:37:34PM +1000, Alexey Kardashevskiy wrote: > The pseries guests do not normally allocate PCI resources and rely on > the system firmware doing so. Furthermore at least at some point in > the past the pseries guests won't even allowed to change BARs, probably > it is still the case for phyp. So since the initial commit we have [1] > which prevents resource reallocation. > > This is not a problem until we want specific BAR alignments, for example, > PAGE_SIZE==64k to make sure we can still map MMIO BARs directly. For > the boot time devices we handle this in SLOF [2] but since QEMU's RTAS > does not allocate BARs, the guest does this instead and does not align > BARs even if Linux is given pci=resource_alignment=16@pci:0:0 as > PCI_PROBE_ONLY makes Linux ignore alignment requests. > > ARM folks added a dial to control PCI_PROBE_ONLY via the device tree [3]. > This makes use of the dial to advertise to the guest that we can handle > BAR reassignments. This limits the change to the latest pseries machine > to avoid old guests explosion. > > We do not remove the flag from [1] as pseries guests are still supported > under phyp so having that removed may cause problems. > > [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/powerpc/platforms/pseries/setup.c?h=v5.1#n773 > [2] https://git.qemu.org/?p=SLOF.git;a=blob;f=board-qemu/slof/pci-phb.fs;h=06729bcf77a0d4e900c527adcd9befe2a269f65d;hb=HEAD#l338 > [3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=f81c11af > Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru> Applied to ppc-for-4.2, thanks. > --- > Changes: > v3: > * moved the default setting to spapr_machine_class_init() > > v2: > * limited the change by a machine version > --- > include/hw/ppc/spapr.h | 1 + > hw/ppc/spapr.c | 9 +++++++++ > 2 files changed, 10 insertions(+) > > diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h > index c79bc6a1232b..ebbd92673b34 100644 > --- a/include/hw/ppc/spapr.h > +++ b/include/hw/ppc/spapr.h > @@ -121,6 +121,7 @@ struct SpaprMachineClass { > bool legacy_irq_allocation; > bool broken_host_serial_model; /* present real host info to the guest */ > bool pre_4_1_migration; /* don't migrate hpt-max-page-size */ > + bool linux_pci_probe; > > void (*phb_placement)(SpaprMachineState *spapr, uint32_t index, > uint64_t *buid, hwaddr *pio, > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c > index 2ee671e55e0c..4bff0cf90d4b 100644 > --- a/hw/ppc/spapr.c > +++ b/hw/ppc/spapr.c > @@ -1164,6 +1164,7 @@ static void spapr_dt_ov5_platform_support(SpaprMachineState *spapr, void *fdt, > static void spapr_dt_chosen(SpaprMachineState *spapr, void *fdt) > { > MachineState *machine = MACHINE(spapr); > + SpaprMachineClass *smc = SPAPR_MACHINE_GET_CLASS(machine); > int chosen; > const char *boot_device = machine->boot_order; > char *stdout_path = spapr_vio_stdout_path(spapr->vio_bus); > @@ -1221,6 +1222,11 @@ static void spapr_dt_chosen(SpaprMachineState *spapr, void *fdt) > _FDT(fdt_setprop_string(fdt, chosen, "stdout-path", stdout_path)); > } > > + /* We can deal with BAR reallocation just fine, advertise it to the guest */ > + if (smc->linux_pci_probe) { > + _FDT(fdt_setprop_cell(fdt, chosen, "linux,pci-probe-only", 0)); > + } > + > spapr_dt_ov5_platform_support(spapr, fdt, chosen); > > g_free(stdout_path); > @@ -4467,6 +4473,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data) > spapr_caps_add_properties(smc, &error_abort); > smc->irq = &spapr_irq_dual; > smc->dr_phb_enabled = true; > + smc->linux_pci_probe = true; > } > > static const TypeInfo spapr_machine_info = { > @@ -4526,12 +4533,14 @@ DEFINE_SPAPR_MACHINE(4_2, "4.2", true); > */ > static void spapr_machine_4_1_class_options(MachineClass *mc) > { > + SpaprMachineClass *smc = SPAPR_MACHINE_CLASS(mc); > static GlobalProperty compat[] = { > /* Only allow 4kiB and 64kiB IOMMU pagesizes */ > { TYPE_SPAPR_PCI_HOST_BRIDGE, "pgsz", "0x11000" }, > }; > > spapr_machine_4_2_class_options(mc); > + smc->linux_pci_probe = false; > compat_props_add(mc->compat_props, compat, G_N_ELEMENTS(compat)); > } >
On Fri, 19 Jul 2019 14:37:34 +1000 Alexey Kardashevskiy <aik@ozlabs.ru> wrote: > The pseries guests do not normally allocate PCI resources and rely on > the system firmware doing so. Furthermore at least at some point in > the past the pseries guests won't even allowed to change BARs, probably > it is still the case for phyp. So since the initial commit we have [1] > which prevents resource reallocation. > > This is not a problem until we want specific BAR alignments, for example, > PAGE_SIZE==64k to make sure we can still map MMIO BARs directly. For > the boot time devices we handle this in SLOF [2] but since QEMU's RTAS > does not allocate BARs, the guest does this instead and does not align > BARs even if Linux is given pci=resource_alignment=16@pci:0:0 as > PCI_PROBE_ONLY makes Linux ignore alignment requests. > > ARM folks added a dial to control PCI_PROBE_ONLY via the device tree [3]. > This makes use of the dial to advertise to the guest that we can handle > BAR reassignments. This limits the change to the latest pseries machine > to avoid old guests explosion. > > We do not remove the flag from [1] as pseries guests are still supported > under phyp so having that removed may cause problems. > > [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/powerpc/platforms/pseries/setup.c?h=v5.1#n773 > [2] https://git.qemu.org/?p=SLOF.git;a=blob;f=board-qemu/slof/pci-phb.fs;h=06729bcf77a0d4e900c527adcd9befe2a269f65d;hb=HEAD#l338 > [3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=f81c11af > Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru> > --- LGTM Reviewed-by: Greg Kurz <groug@kaod.org> Alexey, I only received David's answer in my INBOX... I'm afraid you'll receive a yet another bounce from OVH's MX :-\ Since OVH seems to be ignoring the issue, maybe this could be worked around if you configure your postfix to relay mails to kaod.org through some other relay ? Cheers, -- Greg > Changes: > v3: > * moved the default setting to spapr_machine_class_init() > > v2: > * limited the change by a machine version > --- > include/hw/ppc/spapr.h | 1 + > hw/ppc/spapr.c | 9 +++++++++ > 2 files changed, 10 insertions(+) > > diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h > index c79bc6a1232b..ebbd92673b34 100644 > --- a/include/hw/ppc/spapr.h > +++ b/include/hw/ppc/spapr.h > @@ -121,6 +121,7 @@ struct SpaprMachineClass { > bool legacy_irq_allocation; > bool broken_host_serial_model; /* present real host info to the guest */ > bool pre_4_1_migration; /* don't migrate hpt-max-page-size */ > + bool linux_pci_probe; > > void (*phb_placement)(SpaprMachineState *spapr, uint32_t index, > uint64_t *buid, hwaddr *pio, > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c > index 2ee671e55e0c..4bff0cf90d4b 100644 > --- a/hw/ppc/spapr.c > +++ b/hw/ppc/spapr.c > @@ -1164,6 +1164,7 @@ static void spapr_dt_ov5_platform_support(SpaprMachineState *spapr, void *fdt, > static void spapr_dt_chosen(SpaprMachineState *spapr, void *fdt) > { > MachineState *machine = MACHINE(spapr); > + SpaprMachineClass *smc = SPAPR_MACHINE_GET_CLASS(machine); > int chosen; > const char *boot_device = machine->boot_order; > char *stdout_path = spapr_vio_stdout_path(spapr->vio_bus); > @@ -1221,6 +1222,11 @@ static void spapr_dt_chosen(SpaprMachineState *spapr, void *fdt) > _FDT(fdt_setprop_string(fdt, chosen, "stdout-path", stdout_path)); > } > > + /* We can deal with BAR reallocation just fine, advertise it to the guest */ > + if (smc->linux_pci_probe) { > + _FDT(fdt_setprop_cell(fdt, chosen, "linux,pci-probe-only", 0)); > + } > + > spapr_dt_ov5_platform_support(spapr, fdt, chosen); > > g_free(stdout_path); > @@ -4467,6 +4473,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data) > spapr_caps_add_properties(smc, &error_abort); > smc->irq = &spapr_irq_dual; > smc->dr_phb_enabled = true; > + smc->linux_pci_probe = true; > } > > static const TypeInfo spapr_machine_info = { > @@ -4526,12 +4533,14 @@ DEFINE_SPAPR_MACHINE(4_2, "4.2", true); > */ > static void spapr_machine_4_1_class_options(MachineClass *mc) > { > + SpaprMachineClass *smc = SPAPR_MACHINE_CLASS(mc); > static GlobalProperty compat[] = { > /* Only allow 4kiB and 64kiB IOMMU pagesizes */ > { TYPE_SPAPR_PCI_HOST_BRIDGE, "pgsz", "0x11000" }, > }; > > spapr_machine_4_2_class_options(mc); > + smc->linux_pci_probe = false; > compat_props_add(mc->compat_props, compat, G_N_ELEMENTS(compat)); > } >
diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h index c79bc6a1232b..ebbd92673b34 100644 --- a/include/hw/ppc/spapr.h +++ b/include/hw/ppc/spapr.h @@ -121,6 +121,7 @@ struct SpaprMachineClass { bool legacy_irq_allocation; bool broken_host_serial_model; /* present real host info to the guest */ bool pre_4_1_migration; /* don't migrate hpt-max-page-size */ + bool linux_pci_probe; void (*phb_placement)(SpaprMachineState *spapr, uint32_t index, uint64_t *buid, hwaddr *pio, diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index 2ee671e55e0c..4bff0cf90d4b 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -1164,6 +1164,7 @@ static void spapr_dt_ov5_platform_support(SpaprMachineState *spapr, void *fdt, static void spapr_dt_chosen(SpaprMachineState *spapr, void *fdt) { MachineState *machine = MACHINE(spapr); + SpaprMachineClass *smc = SPAPR_MACHINE_GET_CLASS(machine); int chosen; const char *boot_device = machine->boot_order; char *stdout_path = spapr_vio_stdout_path(spapr->vio_bus); @@ -1221,6 +1222,11 @@ static void spapr_dt_chosen(SpaprMachineState *spapr, void *fdt) _FDT(fdt_setprop_string(fdt, chosen, "stdout-path", stdout_path)); } + /* We can deal with BAR reallocation just fine, advertise it to the guest */ + if (smc->linux_pci_probe) { + _FDT(fdt_setprop_cell(fdt, chosen, "linux,pci-probe-only", 0)); + } + spapr_dt_ov5_platform_support(spapr, fdt, chosen); g_free(stdout_path); @@ -4467,6 +4473,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data) spapr_caps_add_properties(smc, &error_abort); smc->irq = &spapr_irq_dual; smc->dr_phb_enabled = true; + smc->linux_pci_probe = true; } static const TypeInfo spapr_machine_info = { @@ -4526,12 +4533,14 @@ DEFINE_SPAPR_MACHINE(4_2, "4.2", true); */ static void spapr_machine_4_1_class_options(MachineClass *mc) { + SpaprMachineClass *smc = SPAPR_MACHINE_CLASS(mc); static GlobalProperty compat[] = { /* Only allow 4kiB and 64kiB IOMMU pagesizes */ { TYPE_SPAPR_PCI_HOST_BRIDGE, "pgsz", "0x11000" }, }; spapr_machine_4_2_class_options(mc); + smc->linux_pci_probe = false; compat_props_add(mc->compat_props, compat, G_N_ELEMENTS(compat)); }
The pseries guests do not normally allocate PCI resources and rely on the system firmware doing so. Furthermore at least at some point in the past the pseries guests won't even allowed to change BARs, probably it is still the case for phyp. So since the initial commit we have [1] which prevents resource reallocation. This is not a problem until we want specific BAR alignments, for example, PAGE_SIZE==64k to make sure we can still map MMIO BARs directly. For the boot time devices we handle this in SLOF [2] but since QEMU's RTAS does not allocate BARs, the guest does this instead and does not align BARs even if Linux is given pci=resource_alignment=16@pci:0:0 as PCI_PROBE_ONLY makes Linux ignore alignment requests. ARM folks added a dial to control PCI_PROBE_ONLY via the device tree [3]. This makes use of the dial to advertise to the guest that we can handle BAR reassignments. This limits the change to the latest pseries machine to avoid old guests explosion. We do not remove the flag from [1] as pseries guests are still supported under phyp so having that removed may cause problems. [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/powerpc/platforms/pseries/setup.c?h=v5.1#n773 [2] https://git.qemu.org/?p=SLOF.git;a=blob;f=board-qemu/slof/pci-phb.fs;h=06729bcf77a0d4e900c527adcd9befe2a269f65d;hb=HEAD#l338 [3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=f81c11af Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru> --- Changes: v3: * moved the default setting to spapr_machine_class_init() v2: * limited the change by a machine version --- include/hw/ppc/spapr.h | 1 + hw/ppc/spapr.c | 9 +++++++++ 2 files changed, 10 insertions(+)