| Message ID | 20240220154204.29676-1-anisinha@redhat.com |
|---|---|
| State | New |
| Headers | show |
| Series | [v3] pc: q35: Bump max_cpus to 4096 vcpus | expand |
On Tue, Feb 20, 2024 at 09:12:04PM +0530, Ani Sinha wrote: > Since commit f10a570b093e6 ("KVM: x86: Add CONFIG_KVM_MAX_NR_VCPUS to allow up to 4096 vCPUs") > Linux kernel can support upto a maximum number of 4096 vCPUS when MAXSMP is > enabled in the kernel. At present, QEMU has been tested to correctly boot a > linux guest with 4096 vcpus both with edk2 and seabios firmwares. > So bump up the value max_cpus to 4096 for q35 machines versions 9 and newer. > Q35 machines versions 8.2 and older continue to support 1024 maximum vcpus > as before for compatibility reasons. > > If KVM is not able to support the specified number of vcpus, QEMU would > return the following error messages: > > $ ./qemu-system-x86_64 -cpu host -accel kvm -machine q35 -smp 1728 > qemu-system-x86_64: -accel kvm: warning: Number of SMP cpus requested (1728) exceeds the recommended cpus supported by KVM (12) > qemu-system-x86_64: -accel kvm: warning: Number of hotpluggable cpus requested (1728) exceeds the recommended cpus supported by KVM (12) > Number of SMP cpus requested (1728) exceeds the maximum cpus supported by KVM (1024) > > Cc: Daniel P. Berrangé <berrange@redhat.com> > Cc: Igor Mammedov <imammedo@redhat.com> > Cc: Michael S. Tsirkin <mst@redhat.com> > Cc: Julia Suvorova <jusual@redhat.com> > Cc: kraxel@redhat.com > Signed-off-by: Ani Sinha <anisinha@redhat.com> > --- > hw/i386/pc_q35.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > Changelog: > v3: bump up to 4096 vcpus. It has now been tested to work with edk2. > See RH Jira: https://issues.redhat.com/browse/RHEL-22202 That bug indicates a dependancy on a EDK2 patch https://github.com/kraxel/edk2/commit/7a03c17f0f4f4a9003d77db2660c8e087604b2f0 we'll need to rebase the EDK2 ROMs in QEMU to get that included. Meanwhile, plesae at least call out this EDK2 commit as a pre-requisite in the commit message, so people know the EDK2 ROMs in QEMU won't work (yet). > > v2: bump up the vcpu number to 1856. Add failure messages from ekd2 in > the commit description. > > diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c > index d346fa3b1d..ae60e6b919 100644 > --- a/hw/i386/pc_q35.c > +++ b/hw/i386/pc_q35.c > @@ -366,7 +366,7 @@ static void pc_q35_machine_options(MachineClass *m) > m->default_nic = "e1000e"; > m->default_kernel_irqchip_split = false; > m->no_floppy = 1; > - m->max_cpus = 1024; > + m->max_cpus = 4096; > m->no_parallel = !module_object_class_by_name(TYPE_ISA_PARALLEL); > machine_class_allow_dynamic_sysbus_dev(m, TYPE_AMD_IOMMU_DEVICE); > machine_class_allow_dynamic_sysbus_dev(m, TYPE_INTEL_IOMMU_DEVICE); > @@ -387,6 +387,7 @@ static void pc_q35_8_2_machine_options(MachineClass *m) > { > pc_q35_9_0_machine_options(m); > m->alias = NULL; > + m->max_cpus = 1024; > compat_props_add(m->compat_props, hw_compat_8_2, hw_compat_8_2_len); > compat_props_add(m->compat_props, pc_compat_8_2, pc_compat_8_2_len); > } Assuming tweaked commit message Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> With regards, Daniel
On Tue, Feb 20, 2024 at 03:50:57PM +0000, Daniel P. Berrangé wrote: > On Tue, Feb 20, 2024 at 09:12:04PM +0530, Ani Sinha wrote: > > Since commit f10a570b093e6 ("KVM: x86: Add CONFIG_KVM_MAX_NR_VCPUS to allow up to 4096 vCPUs") > > Linux kernel can support upto a maximum number of 4096 vCPUS when MAXSMP is > > enabled in the kernel. At present, QEMU has been tested to correctly boot a > > linux guest with 4096 vcpus both with edk2 and seabios firmwares. > > So bump up the value max_cpus to 4096 for q35 machines versions 9 and newer. > > Q35 machines versions 8.2 and older continue to support 1024 maximum vcpus > > as before for compatibility reasons. > > > > If KVM is not able to support the specified number of vcpus, QEMU would > > return the following error messages: > > > > $ ./qemu-system-x86_64 -cpu host -accel kvm -machine q35 -smp 1728 > > qemu-system-x86_64: -accel kvm: warning: Number of SMP cpus requested (1728) exceeds the recommended cpus supported by KVM (12) > > qemu-system-x86_64: -accel kvm: warning: Number of hotpluggable cpus requested (1728) exceeds the recommended cpus supported by KVM (12) > > Number of SMP cpus requested (1728) exceeds the maximum cpus supported by KVM (1024) > > > > Cc: Daniel P. Berrangé <berrange@redhat.com> > > Cc: Igor Mammedov <imammedo@redhat.com> > > Cc: Michael S. Tsirkin <mst@redhat.com> > > Cc: Julia Suvorova <jusual@redhat.com> > > Cc: kraxel@redhat.com > > Signed-off-by: Ani Sinha <anisinha@redhat.com> > > --- > > hw/i386/pc_q35.c | 3 ++- > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > Changelog: > > v3: bump up to 4096 vcpus. It has now been tested to work with edk2. > > See RH Jira: https://issues.redhat.com/browse/RHEL-22202 > > That bug indicates a dependancy on a EDK2 patch > > https://github.com/kraxel/edk2/commit/7a03c17f0f4f4a9003d77db2660c8e087604b2f0 > > we'll need to rebase the EDK2 ROMs in QEMU to get that included. Which will btw take a while. edk2 is in freeze for the 2024-02 release right now, I expect the changes land upstream shortly thereafter and will be part of the 2024-05 release. So end may / early june would be the time when rebasing to release, or somewhen in march or april when we rebase to a git snapshot ... > Meanwhile, plesae at least call out this EDK2 commit as a > pre-requisite in the commit message, so people know the > EDK2 ROMs in QEMU won't work (yet). That surely makes sense. Oh, and it's more than just that one commit. I don't think it makes sense to compile a list of commits given this is a moving target (upstream review is in progress right now). take care, Gerd
On Tue, Feb 20, 2024 at 05:16:37PM +0100, Gerd Hoffmann wrote: > On Tue, Feb 20, 2024 at 03:50:57PM +0000, Daniel P. Berrangé wrote: > > On Tue, Feb 20, 2024 at 09:12:04PM +0530, Ani Sinha wrote: > > > Since commit f10a570b093e6 ("KVM: x86: Add CONFIG_KVM_MAX_NR_VCPUS to allow up to 4096 vCPUs") > > > Linux kernel can support upto a maximum number of 4096 vCPUS when MAXSMP is > > > enabled in the kernel. At present, QEMU has been tested to correctly boot a > > > linux guest with 4096 vcpus both with edk2 and seabios firmwares. > > > So bump up the value max_cpus to 4096 for q35 machines versions 9 and newer. > > > Q35 machines versions 8.2 and older continue to support 1024 maximum vcpus > > > as before for compatibility reasons. > > > > > > If KVM is not able to support the specified number of vcpus, QEMU would > > > return the following error messages: > > > > > > $ ./qemu-system-x86_64 -cpu host -accel kvm -machine q35 -smp 1728 > > > qemu-system-x86_64: -accel kvm: warning: Number of SMP cpus requested (1728) exceeds the recommended cpus supported by KVM (12) > > > qemu-system-x86_64: -accel kvm: warning: Number of hotpluggable cpus requested (1728) exceeds the recommended cpus supported by KVM (12) > > > Number of SMP cpus requested (1728) exceeds the maximum cpus supported by KVM (1024) > > > > > > Cc: Daniel P. Berrangé <berrange@redhat.com> > > > Cc: Igor Mammedov <imammedo@redhat.com> > > > Cc: Michael S. Tsirkin <mst@redhat.com> > > > Cc: Julia Suvorova <jusual@redhat.com> > > > Cc: kraxel@redhat.com > > > Signed-off-by: Ani Sinha <anisinha@redhat.com> > > > --- > > > hw/i386/pc_q35.c | 3 ++- > > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > > > Changelog: > > > v3: bump up to 4096 vcpus. It has now been tested to work with edk2. > > > See RH Jira: https://issues.redhat.com/browse/RHEL-22202 > > > > That bug indicates a dependancy on a EDK2 patch > > > > https://github.com/kraxel/edk2/commit/7a03c17f0f4f4a9003d77db2660c8e087604b2f0 > > > > we'll need to rebase the EDK2 ROMs in QEMU to get that included. > > Which will btw take a while. edk2 is in freeze for the 2024-02 release > right now, I expect the changes land upstream shortly thereafter and > will be part of the 2024-05 release. So end may / early june would be > the time when rebasing to release, or somewhen in march or april when we > rebase to a git snapshot ... > > > Meanwhile, plesae at least call out this EDK2 commit as a > > pre-requisite in the commit message, so people know the > > EDK2 ROMs in QEMU won't work (yet). > > That surely makes sense. > > Oh, and it's more than just that one commit. I don't think it makes > sense to compile a list of commits given this is a moving target > (upstream review is in progress right now). Ok, lets just mention something hand-wavy like "pending various upstream EDK2 fixes which will probably be in the 2024-05 release" With regards, Daniel
> On 20-Feb-2024, at 21:46, Gerd Hoffmann <kraxel@redhat.com> wrote: > > On Tue, Feb 20, 2024 at 03:50:57PM +0000, Daniel P. Berrangé wrote: >> On Tue, Feb 20, 2024 at 09:12:04PM +0530, Ani Sinha wrote: >>> Since commit f10a570b093e6 ("KVM: x86: Add CONFIG_KVM_MAX_NR_VCPUS to allow up to 4096 vCPUs") >>> Linux kernel can support upto a maximum number of 4096 vCPUS when MAXSMP is >>> enabled in the kernel. At present, QEMU has been tested to correctly boot a >>> linux guest with 4096 vcpus both with edk2 and seabios firmwares. >>> So bump up the value max_cpus to 4096 for q35 machines versions 9 and newer. >>> Q35 machines versions 8.2 and older continue to support 1024 maximum vcpus >>> as before for compatibility reasons. >>> >>> If KVM is not able to support the specified number of vcpus, QEMU would >>> return the following error messages: >>> >>> $ ./qemu-system-x86_64 -cpu host -accel kvm -machine q35 -smp 1728 >>> qemu-system-x86_64: -accel kvm: warning: Number of SMP cpus requested (1728) exceeds the recommended cpus supported by KVM (12) >>> qemu-system-x86_64: -accel kvm: warning: Number of hotpluggable cpus requested (1728) exceeds the recommended cpus supported by KVM (12) >>> Number of SMP cpus requested (1728) exceeds the maximum cpus supported by KVM (1024) >>> >>> Cc: Daniel P. Berrangé <berrange@redhat.com> >>> Cc: Igor Mammedov <imammedo@redhat.com> >>> Cc: Michael S. Tsirkin <mst@redhat.com> >>> Cc: Julia Suvorova <jusual@redhat.com> >>> Cc: kraxel@redhat.com >>> Signed-off-by: Ani Sinha <anisinha@redhat.com> >>> --- >>> hw/i386/pc_q35.c | 3 ++- >>> 1 file changed, 2 insertions(+), 1 deletion(-) >>> >>> Changelog: >>> v3: bump up to 4096 vcpus. It has now been tested to work with edk2. >>> See RH Jira: https://issues.redhat.com/browse/RHEL-22202 >> >> That bug indicates a dependancy on a EDK2 patch >> >> https://github.com/kraxel/edk2/commit/7a03c17f0f4f4a9003d77db2660c8e087604b2f0 >> >> we'll need to rebase the EDK2 ROMs in QEMU to get that included. > > Which will btw take a while. edk2 is in freeze for the 2024-02 release > right now, I expect the changes land upstream shortly thereafter and > will be part of the 2024-05 release. So end may / early june would be > the time when rebasing to release, or somewhen in march or april when we > rebase to a git snapshot ... > >> Meanwhile, plesae at least call out this EDK2 commit as a >> pre-requisite in the commit message, so people know the >> EDK2 ROMs in QEMU won't work (yet). > > That surely makes sense. > > Oh, and it's more than just that one commit. I don't think it makes > sense to compile a list of commits given this is a moving target > (upstream review is in progress right now). Looking at the edk2 GH, are these the PR that are waiting for upstream review/merge that relate to vcpu scaling? https://github.com/tianocore/edk2/pull/5375 https://github.com/tianocore/edk2/pull/5327
Hi, > Looking at the edk2 GH, are these the PR that are waiting for upstream review/merge that relate to vcpu scaling? > > https://github.com/tianocore/edk2/pull/5375 > https://github.com/tianocore/edk2/pull/5327 These are draft MRs for running CI. The current devel branches are: https://github.com/kraxel/edk2/tree/devel/many-vcpus https://github.com/kraxel/edk2/tree/devel/many-vcpus-mpinitlib All of them will expire at some point though, so I don't think it is a good idea to include them in the commit message. They will point into nowhere in a year or so. take care, Gerd
diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c index d346fa3b1d..ae60e6b919 100644 --- a/hw/i386/pc_q35.c +++ b/hw/i386/pc_q35.c @@ -366,7 +366,7 @@ static void pc_q35_machine_options(MachineClass *m) m->default_nic = "e1000e"; m->default_kernel_irqchip_split = false; m->no_floppy = 1; - m->max_cpus = 1024; + m->max_cpus = 4096; m->no_parallel = !module_object_class_by_name(TYPE_ISA_PARALLEL); machine_class_allow_dynamic_sysbus_dev(m, TYPE_AMD_IOMMU_DEVICE); machine_class_allow_dynamic_sysbus_dev(m, TYPE_INTEL_IOMMU_DEVICE); @@ -387,6 +387,7 @@ static void pc_q35_8_2_machine_options(MachineClass *m) { pc_q35_9_0_machine_options(m); m->alias = NULL; + m->max_cpus = 1024; compat_props_add(m->compat_props, hw_compat_8_2, hw_compat_8_2_len); compat_props_add(m->compat_props, pc_compat_8_2, pc_compat_8_2_len); }
Since commit f10a570b093e6 ("KVM: x86: Add CONFIG_KVM_MAX_NR_VCPUS to allow up to 4096 vCPUs") Linux kernel can support upto a maximum number of 4096 vCPUS when MAXSMP is enabled in the kernel. At present, QEMU has been tested to correctly boot a linux guest with 4096 vcpus both with edk2 and seabios firmwares. So bump up the value max_cpus to 4096 for q35 machines versions 9 and newer. Q35 machines versions 8.2 and older continue to support 1024 maximum vcpus as before for compatibility reasons. If KVM is not able to support the specified number of vcpus, QEMU would return the following error messages: $ ./qemu-system-x86_64 -cpu host -accel kvm -machine q35 -smp 1728 qemu-system-x86_64: -accel kvm: warning: Number of SMP cpus requested (1728) exceeds the recommended cpus supported by KVM (12) qemu-system-x86_64: -accel kvm: warning: Number of hotpluggable cpus requested (1728) exceeds the recommended cpus supported by KVM (12) Number of SMP cpus requested (1728) exceeds the maximum cpus supported by KVM (1024) Cc: Daniel P. Berrangé <berrange@redhat.com> Cc: Igor Mammedov <imammedo@redhat.com> Cc: Michael S. Tsirkin <mst@redhat.com> Cc: Julia Suvorova <jusual@redhat.com> Cc: kraxel@redhat.com Signed-off-by: Ani Sinha <anisinha@redhat.com> --- hw/i386/pc_q35.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) Changelog: v3: bump up to 4096 vcpus. It has now been tested to work with edk2. See RH Jira: https://issues.redhat.com/browse/RHEL-22202 v2: bump up the vcpu number to 1856. Add failure messages from ekd2 in the commit description.