| Message ID | 20240202215332.118728-4-david@redhat.com |
|---|---|
| State | New |
| Headers | show |
| Series | libvhost-user: support more memslots and cleanup memslot handling code | expand |
On Fri, Feb 2, 2024 at 4:54 PM David Hildenbrand <david@redhat.com> wrote: > > Let's support up to 509 mem slots, just like vhost in the kernel usually > does and the rust vhost-user implementation recently [1] started doing. > This is required to properly support memory hotplug, either using > multiple DIMMs (ACPI supports up to 256) or using virtio-mem. > > The 509 used to be the KVM limit, it supported 512, but 3 were > used for internal purposes. Currently, KVM supports more than 512, but > it usually doesn't make use of more than ~260 (i.e., 256 DIMMs + boot > memory), except when other memory devices like PCI devices with BARs are > used. So, 509 seems to work well for vhost in the kernel. > > Details can be found in the QEMU change that made virtio-mem consume > up to 256 mem slots across all virtio-mem devices. [2] > > 509 mem slots implies 509 VMAs/mappings in the worst case (even though, > in practice with virtio-mem we won't be seeing more than ~260 in most > setups). > > With max_map_count under Linux defaulting to 64k, 509 mem slots > still correspond to less than 1% of the maximum number of mappings. > There are plenty left for the application to consume. > > [1] https://github.com/rust-vmm/vhost/pull/224 > [2] https://lore.kernel.org/all/20230926185738.277351-1-david@redhat.com/ > > Signed-off-by: David Hildenbrand <david@redhat.com> Reviewed-by: Raphael Norwitz <raphael@enfabrica.net> > --- > subprojects/libvhost-user/libvhost-user.h | 8 +++++--- > 1 file changed, 5 insertions(+), 3 deletions(-) > > diff --git a/subprojects/libvhost-user/libvhost-user.h b/subprojects/libvhost-user/libvhost-user.h > index c882b4e3a2..deb40e77b3 100644 > --- a/subprojects/libvhost-user/libvhost-user.h > +++ b/subprojects/libvhost-user/libvhost-user.h > @@ -31,10 +31,12 @@ > #define VHOST_MEMORY_BASELINE_NREGIONS 8 > > /* > - * Set a reasonable maximum number of ram slots, which will be supported by > - * any architecture. > + * vhost in the kernel usually supports 509 mem slots. 509 used to be the > + * KVM limit, it supported 512, but 3 were used for internal purposes. This > + * limit is sufficient to support many DIMMs and virtio-mem in > + * "dynamic-memslots" mode. > */ > -#define VHOST_USER_MAX_RAM_SLOTS 32 > +#define VHOST_USER_MAX_RAM_SLOTS 509 > > #define VHOST_USER_HDR_SIZE offsetof(VhostUserMsg, payload.u64) > > -- > 2.43.0 > >
diff --git a/subprojects/libvhost-user/libvhost-user.h b/subprojects/libvhost-user/libvhost-user.h index c882b4e3a2..deb40e77b3 100644 --- a/subprojects/libvhost-user/libvhost-user.h +++ b/subprojects/libvhost-user/libvhost-user.h @@ -31,10 +31,12 @@ #define VHOST_MEMORY_BASELINE_NREGIONS 8 /* - * Set a reasonable maximum number of ram slots, which will be supported by - * any architecture. + * vhost in the kernel usually supports 509 mem slots. 509 used to be the + * KVM limit, it supported 512, but 3 were used for internal purposes. This + * limit is sufficient to support many DIMMs and virtio-mem in + * "dynamic-memslots" mode. */ -#define VHOST_USER_MAX_RAM_SLOTS 32 +#define VHOST_USER_MAX_RAM_SLOTS 509 #define VHOST_USER_HDR_SIZE offsetof(VhostUserMsg, payload.u64)
Let's support up to 509 mem slots, just like vhost in the kernel usually does and the rust vhost-user implementation recently [1] started doing. This is required to properly support memory hotplug, either using multiple DIMMs (ACPI supports up to 256) or using virtio-mem. The 509 used to be the KVM limit, it supported 512, but 3 were used for internal purposes. Currently, KVM supports more than 512, but it usually doesn't make use of more than ~260 (i.e., 256 DIMMs + boot memory), except when other memory devices like PCI devices with BARs are used. So, 509 seems to work well for vhost in the kernel. Details can be found in the QEMU change that made virtio-mem consume up to 256 mem slots across all virtio-mem devices. [2] 509 mem slots implies 509 VMAs/mappings in the worst case (even though, in practice with virtio-mem we won't be seeing more than ~260 in most setups). With max_map_count under Linux defaulting to 64k, 509 mem slots still correspond to less than 1% of the maximum number of mappings. There are plenty left for the application to consume. [1] https://github.com/rust-vmm/vhost/pull/224 [2] https://lore.kernel.org/all/20230926185738.277351-1-david@redhat.com/ Signed-off-by: David Hildenbrand <david@redhat.com> --- subprojects/libvhost-user/libvhost-user.h | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)