| Message ID | 20240910175809.2135596-1-david@redhat.com |
|---|---|
| Headers | show |
| Series | s390x: virtio-mem support | expand |
On Tue, Sep 10, 2024 at 07:57:55PM +0200, David Hildenbrand wrote: > This series is based on: > [PATCH v2] virtio: kconfig: memory devices are PCI only [1] > > I finally found the time (IOW forced myself) to finish virtio-mem > support on s390x. The last RFC was from 2020, so I won't talk about > what changed -- a lot changed in the meantime :) > > There is really not much left to do on s390x, because virtio-mem already > implements most things we need today (e.g., early-migration, > unplugged-inaccessible). The biggest part of this series is just doing what > we do with virtio-pci, wiring it up in the machine hotplug handler and ... > well, messing with the physical memory layout where we can now exceed > initial RAM size and have sparsity (memory holes). > > I tested a lot of things, including: > * Memory hotplug/unplug > * Device hotplug/unplug > * System resets / reboots > * Migrate to/from file (including storage attributes under KVM) > * Basic live migration > * Basic postcopy live migration > > More details on how to use it on s390x -- which is pretty much how > we use it on other architectures, except > s/virtio-mem-pci/virtio-mem-ccw/ --- is in the last patch. > > This series introduces a new diag(500) "STORAGE LIMIT" subcode that will > be documented at [2] once this+kernel part go upstream. > > There are not many s390x-specific virtio-mem future work items, except: > * Storage attribute migration might be improved > * We might want to reset storage attributes of unplugged memory > (might or might not be required for upcoming page table reclaim in > Linux; TBD) I don't see anything needing virtio specific here, let me know if I missed anything. A quick look is fine so I guess you can add Acked-by: Michael S. Tsirkin <mst@redhat.com> > I'll send out the kernel driver bits soon. > > [1] https://lkml.kernel.org/r/20240906101658.514470-1-pbonzini@redhat.com > [2] https://gitlab.com/davidhildenbrand/s390x-os-virt-spec > > Cc: Paolo Bonzini <pbonzini@redhat.com> > Cc: Thomas Huth <thuth@redhat.com> > Cc: Halil Pasic <pasic@linux.ibm.com> > Cc: Christian Borntraeger <borntraeger@linux.ibm.com> > Cc: Eric Farman <farman@linux.ibm.com> > Cc: Richard Henderson <richard.henderson@linaro.org> > Cc: David Hildenbrand <david@redhat.com> > Cc: Ilya Leoshkevich <iii@linux.ibm.com> > Cc: Janosch Frank <frankja@linux.ibm.com> > Cc: "Michael S. Tsirkin" <mst@redhat.com> > Cc: Cornelia Huck <cohuck@redhat.com> > > David Hildenbrand (14): > s390x/s390-virtio-ccw: don't crash on weird RAM sizes > s390x/s390-virtio-hcall: remove hypercall registration mechanism > s390x/s390-virtio-hcall: prepare for more diag500 hypercalls > s390x: rename s390-virtio-hcall* to s390-hypercall* > s390x/s390-virtio-ccw: move setting the maximum guest size from sclp > to machine code > s390x: introduce s390_get_memory_limit() > s390x/s390-hypercall: introduce DIAG500 STORAGE_LIMIT > s390x/s390-stattrib-kvm: prepare memory devices and sparse memory > layouts > s390x/s390-skeys: prepare for memory devices > s390x/pv: check initial, not maximum RAM size > s390x/s390-virtio-ccw: prepare for memory devices > s390x: introduce s390_get_max_pagesize() > s390x/virtio-ccw: add support for virtio based memory devices > s390x: virtio-mem support > > MAINTAINERS | 4 + > hw/s390x/Kconfig | 1 + > hw/s390x/meson.build | 4 +- > hw/s390x/s390-hypercall.c | 77 +++++++++++ > hw/s390x/s390-hypercall.h | 25 ++++ > hw/s390x/s390-skeys.c | 4 +- > hw/s390x/s390-stattrib-kvm.c | 63 +++++---- > hw/s390x/s390-virtio-ccw.c | 143 +++++++++++++-------- > hw/s390x/s390-virtio-hcall.c | 41 ------ > hw/s390x/s390-virtio-hcall.h | 25 ---- > hw/s390x/sclp.c | 17 +-- > hw/s390x/virtio-ccw-md.c | 153 ++++++++++++++++++++++ > hw/s390x/virtio-ccw-md.h | 44 +++++++ > hw/s390x/virtio-ccw-mem.c | 226 +++++++++++++++++++++++++++++++++ > hw/s390x/virtio-ccw-mem.h | 34 +++++ > hw/virtio/Kconfig | 1 + > hw/virtio/virtio-mem.c | 4 +- > target/s390x/cpu-sysemu.c | 35 ++++- > target/s390x/cpu.h | 2 + > target/s390x/kvm/kvm.c | 12 +- > target/s390x/kvm/pv.c | 2 +- > target/s390x/tcg/misc_helper.c | 6 +- > 22 files changed, 746 insertions(+), 177 deletions(-) > create mode 100644 hw/s390x/s390-hypercall.c > create mode 100644 hw/s390x/s390-hypercall.h > delete mode 100644 hw/s390x/s390-virtio-hcall.c > delete mode 100644 hw/s390x/s390-virtio-hcall.h > create mode 100644 hw/s390x/virtio-ccw-md.c > create mode 100644 hw/s390x/virtio-ccw-md.h > create mode 100644 hw/s390x/virtio-ccw-mem.c > create mode 100644 hw/s390x/virtio-ccw-mem.h > > -- > 2.46.0
On 10.09.24 20:33, Michael S. Tsirkin wrote: > On Tue, Sep 10, 2024 at 07:57:55PM +0200, David Hildenbrand wrote: >> This series is based on: >> [PATCH v2] virtio: kconfig: memory devices are PCI only [1] >> >> I finally found the time (IOW forced myself) to finish virtio-mem >> support on s390x. The last RFC was from 2020, so I won't talk about >> what changed -- a lot changed in the meantime :) >> >> There is really not much left to do on s390x, because virtio-mem already >> implements most things we need today (e.g., early-migration, >> unplugged-inaccessible). The biggest part of this series is just doing what >> we do with virtio-pci, wiring it up in the machine hotplug handler and ... >> well, messing with the physical memory layout where we can now exceed >> initial RAM size and have sparsity (memory holes). >> >> I tested a lot of things, including: >> * Memory hotplug/unplug >> * Device hotplug/unplug >> * System resets / reboots >> * Migrate to/from file (including storage attributes under KVM) >> * Basic live migration >> * Basic postcopy live migration >> >> More details on how to use it on s390x -- which is pretty much how >> we use it on other architectures, except >> s/virtio-mem-pci/virtio-mem-ccw/ --- is in the last patch. >> >> This series introduces a new diag(500) "STORAGE LIMIT" subcode that will >> be documented at [2] once this+kernel part go upstream. >> >> There are not many s390x-specific virtio-mem future work items, except: >> * Storage attribute migration might be improved >> * We might want to reset storage attributes of unplugged memory >> (might or might not be required for upcoming page table reclaim in >> Linux; TBD) > > > I don't see anything needing virtio specific here, let me know if > I missed anything. No, it's really just wiring up virtio-mem. > A quick look is fine so I guess you can add > Acked-by: Michael S. Tsirkin <mst@redhat.com> Thanks!
On 9/10/24 7:57 PM, David Hildenbrand wrote: > This series introduces a new diag(500) "STORAGE LIMIT" subcode that will > be documented at [2] once this+kernel part go upstream. Why not in Documentation/virt/kvm/s390/? s390-diag.rst is very similar already. I'd rather have it in a shared and bigger repo than in your personal gitlab one. Maybe there's a space somewhere in QEMU or the Virtio team's repos that would be a good fit if the kernel's docu isn't the right place?
On 11.09.24 13:49, Janosch Frank wrote: > On 9/10/24 7:57 PM, David Hildenbrand wrote: >> This series introduces a new diag(500) "STORAGE LIMIT" subcode that will >> be documented at [2] once this+kernel part go upstream. > > Why not in Documentation/virt/kvm/s390/? > s390-diag.rst is very similar already. I can document it anywhere people fancy, really I don't care. But this has some history. The last time we discussed it [1] there was: "Regardless what we end up with, this needs to be specified somewhere(tm)." from Conny "It must be well defined and easy to find also for kernel developers who actually have to care about memory detection code :)" from Heiko. And then "the kernel's s390-diag.rst should also point to it ... Nothing as complicated as an OASIS spec, but maybe a git??b project?" from Conny > > I'd rather have it in a shared and bigger repo than in your personal > gitlab one. Maybe there's a space somewhere in QEMU or the Virtio team's > repos that would be a good fit if the kernel's docu isn't the right place? At this point, outside of kernel/QEMU feels like the right thing to do. Conny is already a co-maintainer of my "personal" (;)) gitlab. And now I realize that I CCed Heiko on the Linux series but not the QEMU series. My bad. [1] https://lore.kernel.org/all/20200727114819.3f816010.cohuck@redhat.com/
> > > > I'd rather have it in a shared and bigger repo than in your personal > > gitlab one. Maybe there's a space somewhere in QEMU or the Virtio team's > > repos that would be a good fit if the kernel's docu isn't the right place? > > At this point, outside of kernel/QEMU feels like the right thing to do. > Conny is already a co-maintainer of my "personal" (;)) gitlab. > > > And now I realize that I CCed Heiko on the Linux series but not the QEMU > series. My bad. > > [1] https://lore.kernel.org/all/20200727114819.3f816010.cohuck@redhat.com/ No prob. Or if you want it in virtio spec, that's also fine.
On Wed, Sep 11 2024, "Michael S. Tsirkin" <mst@redhat.com> wrote: >> > >> > I'd rather have it in a shared and bigger repo than in your personal >> > gitlab one. Maybe there's a space somewhere in QEMU or the Virtio team's >> > repos that would be a good fit if the kernel's docu isn't the right place? >> >> At this point, outside of kernel/QEMU feels like the right thing to do. >> Conny is already a co-maintainer of my "personal" (;)) gitlab. >> >> >> And now I realize that I CCed Heiko on the Linux series but not the QEMU >> series. My bad. >> >> [1] https://lore.kernel.org/all/20200727114819.3f816010.cohuck@redhat.com/ > > > No prob. Or if you want it in virtio spec, that's also fine. The virtio spec makes sense for documenting things needed to implement it, but what I liked about the gitlab project is that it tries to go into more s390-specific aspects (that feel a bit out of scope for the virtio spec), and that it provides a place to document non-virtio related interfaces. Anyway, if we want to proceed with the gitlab project, would it make sense to create an org for it, so that it doesn't look like David's personal project? In any case, while I'm happy to stay on, I'm not that involved with s390 anymore, and it might make sense to add more people to it.
On 11.09.24 17:38, Cornelia Huck wrote: > On Wed, Sep 11 2024, "Michael S. Tsirkin" <mst@redhat.com> wrote: > >>>> >>>> I'd rather have it in a shared and bigger repo than in your personal >>>> gitlab one. Maybe there's a space somewhere in QEMU or the Virtio team's >>>> repos that would be a good fit if the kernel's docu isn't the right place? >>> >>> At this point, outside of kernel/QEMU feels like the right thing to do. >>> Conny is already a co-maintainer of my "personal" (;)) gitlab. >>> >>> >>> And now I realize that I CCed Heiko on the Linux series but not the QEMU >>> series. My bad. >>> >>> [1] https://lore.kernel.org/all/20200727114819.3f816010.cohuck@redhat.com/ >> >> >> No prob. Or if you want it in virtio spec, that's also fine. > > The virtio spec makes sense for documenting things needed to implement > it, but what I liked about the gitlab project is that it tries to go > into more s390-specific aspects (that feel a bit out of scope for the > virtio spec), and that it provides a place to document non-virtio > related interfaces. > > Anyway, if we want to proceed with the gitlab project, would it make > sense to create an org for it, so that it doesn't look like David's > personal project? I had that in mind as well, I can move the project. > In any case, while I'm happy to stay on, I'm not that > involved with s390 anymore, and it might make sense to add more people > to it. Indeed!
This series is based on: [PATCH v2] virtio: kconfig: memory devices are PCI only [1] I finally found the time (IOW forced myself) to finish virtio-mem support on s390x. The last RFC was from 2020, so I won't talk about what changed -- a lot changed in the meantime :) There is really not much left to do on s390x, because virtio-mem already implements most things we need today (e.g., early-migration, unplugged-inaccessible). The biggest part of this series is just doing what we do with virtio-pci, wiring it up in the machine hotplug handler and ... well, messing with the physical memory layout where we can now exceed initial RAM size and have sparsity (memory holes). I tested a lot of things, including: * Memory hotplug/unplug * Device hotplug/unplug * System resets / reboots * Migrate to/from file (including storage attributes under KVM) * Basic live migration * Basic postcopy live migration More details on how to use it on s390x -- which is pretty much how we use it on other architectures, except s/virtio-mem-pci/virtio-mem-ccw/ --- is in the last patch. This series introduces a new diag(500) "STORAGE LIMIT" subcode that will be documented at [2] once this+kernel part go upstream. There are not many s390x-specific virtio-mem future work items, except: * Storage attribute migration might be improved * We might want to reset storage attributes of unplugged memory (might or might not be required for upcoming page table reclaim in Linux; TBD) I'll send out the kernel driver bits soon. [1] https://lkml.kernel.org/r/20240906101658.514470-1-pbonzini@redhat.com [2] https://gitlab.com/davidhildenbrand/s390x-os-virt-spec Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Thomas Huth <thuth@redhat.com> Cc: Halil Pasic <pasic@linux.ibm.com> Cc: Christian Borntraeger <borntraeger@linux.ibm.com> Cc: Eric Farman <farman@linux.ibm.com> Cc: Richard Henderson <richard.henderson@linaro.org> Cc: David Hildenbrand <david@redhat.com> Cc: Ilya Leoshkevich <iii@linux.ibm.com> Cc: Janosch Frank <frankja@linux.ibm.com> Cc: "Michael S. Tsirkin" <mst@redhat.com> Cc: Cornelia Huck <cohuck@redhat.com> David Hildenbrand (14): s390x/s390-virtio-ccw: don't crash on weird RAM sizes s390x/s390-virtio-hcall: remove hypercall registration mechanism s390x/s390-virtio-hcall: prepare for more diag500 hypercalls s390x: rename s390-virtio-hcall* to s390-hypercall* s390x/s390-virtio-ccw: move setting the maximum guest size from sclp to machine code s390x: introduce s390_get_memory_limit() s390x/s390-hypercall: introduce DIAG500 STORAGE_LIMIT s390x/s390-stattrib-kvm: prepare memory devices and sparse memory layouts s390x/s390-skeys: prepare for memory devices s390x/pv: check initial, not maximum RAM size s390x/s390-virtio-ccw: prepare for memory devices s390x: introduce s390_get_max_pagesize() s390x/virtio-ccw: add support for virtio based memory devices s390x: virtio-mem support MAINTAINERS | 4 + hw/s390x/Kconfig | 1 + hw/s390x/meson.build | 4 +- hw/s390x/s390-hypercall.c | 77 +++++++++++ hw/s390x/s390-hypercall.h | 25 ++++ hw/s390x/s390-skeys.c | 4 +- hw/s390x/s390-stattrib-kvm.c | 63 +++++---- hw/s390x/s390-virtio-ccw.c | 143 +++++++++++++-------- hw/s390x/s390-virtio-hcall.c | 41 ------ hw/s390x/s390-virtio-hcall.h | 25 ---- hw/s390x/sclp.c | 17 +-- hw/s390x/virtio-ccw-md.c | 153 ++++++++++++++++++++++ hw/s390x/virtio-ccw-md.h | 44 +++++++ hw/s390x/virtio-ccw-mem.c | 226 +++++++++++++++++++++++++++++++++ hw/s390x/virtio-ccw-mem.h | 34 +++++ hw/virtio/Kconfig | 1 + hw/virtio/virtio-mem.c | 4 +- target/s390x/cpu-sysemu.c | 35 ++++- target/s390x/cpu.h | 2 + target/s390x/kvm/kvm.c | 12 +- target/s390x/kvm/pv.c | 2 +- target/s390x/tcg/misc_helper.c | 6 +- 22 files changed, 746 insertions(+), 177 deletions(-) create mode 100644 hw/s390x/s390-hypercall.c create mode 100644 hw/s390x/s390-hypercall.h delete mode 100644 hw/s390x/s390-virtio-hcall.c delete mode 100644 hw/s390x/s390-virtio-hcall.h create mode 100644 hw/s390x/virtio-ccw-md.c create mode 100644 hw/s390x/virtio-ccw-md.h create mode 100644 hw/s390x/virtio-ccw-mem.c create mode 100644 hw/s390x/virtio-ccw-mem.h