Message ID | 20230908124534.25027-1-twiederh@redhat.com |
---|---|
Headers | show |
Series | Generate x86 cpu features | expand |
On Fri, 8 Sep 2023 14:45:24 +0200 Tim Wiederhake <twiederh@redhat.com> wrote: > Synchronizing the list of cpu features and models with qemu is a recurring > task in libvirt. For x86, this is done by reading qom-list-properties for > max-x86_64-cpu and manually filtering out everthing that does not look like > a feature name, as well as parsing target/i386/cpu.c for cpu models. modulo fixing typos/name conflicts in 1st 3 patches, I don't think it's a great idea for libvirt (or any other user) to parse QEMU source (whether it's C code or yaml) or other way around for users to influence QEMU internals. QEMU does provides QMP interface for introspection of CPU models and it should be used as a mean to discover new CPU models as well as new feature names. If something is missing there we should fix it up to make it usable for libvirt. > This is a flawed, tedious and error-prone procedure. Ideally, qemu > and libvirt would query a common source for cpu feature and model > related information. Meanwhile, converting this information into an easier > to parse format would help libvirt a lot. > > This patch series converts the cpu feature information present in > target/i386/cpu.c (`feature_word_info`) into a yaml file and adds a > script to generate the c code from this data. > A patch set to convert the cpu model data (`builtin_x86_defs`) in the > same way will follow. while theoretically you can parse feature_word_info with some sort of reliability, you can't do that reliably with CPU model definitions in builtin_x86_defs as the later is just a skeleton. Actual CPU models definitions are a function of used machine type, accelerator and host kernel. > > v1: https://lists.nongnu.org/archive/html/qemu-devel/2023-08/msg02005.html > > Changes since v1: > * Incorporated changes from > https://lists.nongnu.org/archive/html/qemu-devel/2023-08/msg04241.html. > * Changed data format from xml to yaml, as proposed in > https://lists.nongnu.org/archive/html/qemu-devel/2023-09/msg01033.html. > Using json has some drawbacks, see > https://lists.nongnu.org/archive/html/qemu-devel/2023-08/msg03384.html. > * Rebased on top of current master. Features added in the meantime: > amx-complex and vmx-enable-user-wait-pause > * Split up the reformatting of feature_word_info.c.inc to make it easier > to review. These patches could be squashed together before merging. > > Tim Wiederhake (10): > target/i386: Add missing feature names in FEAT_VMX_EPT_VPID_CAPS > target/i386: Fix feature names in FEAT_VMX_EPT_VPID_CAPS > target/i386: Fix duplicated feature name in FEAT_KVM > target/i386: Split out feature_word_info > target/i386: Translate feature_word_info to yaml > target/i386: Format feature_word_info.c.inc: Remove comments > target/i386: Format feature_word_info.c.inc: feat_names > target/i386: Format feature_word_info.c.inc: Unfold cpuid member > target/i386: Format feature_word_info.c.inc: Whitespaces and trailing > commas > target/i386: Autogenerate feature_word_info.c.inc > > target/i386/cpu.c | 677 +----------------- > target/i386/feature_word_info.c.inc | 1030 +++++++++++++++++++++++++++ > target/i386/feature_word_info.py | 62 ++ > target/i386/feature_word_info.yaml | 697 ++++++++++++++++++ > 4 files changed, 1790 insertions(+), 676 deletions(-) > create mode 100644 target/i386/feature_word_info.c.inc > create mode 100755 target/i386/feature_word_info.py > create mode 100644 target/i386/feature_word_info.yaml >
On Fri, Sep 08, 2023 at 04:48:46PM +0200, Igor Mammedov wrote: > On Fri, 8 Sep 2023 14:45:24 +0200 > Tim Wiederhake <twiederh@redhat.com> wrote: > > > Synchronizing the list of cpu features and models with qemu is a recurring > > task in libvirt. For x86, this is done by reading qom-list-properties for > > max-x86_64-cpu and manually filtering out everthing that does not look like > > a feature name, as well as parsing target/i386/cpu.c for cpu models. > > modulo fixing typos/name conflicts in 1st 3 patches, > > I don't think it's a great idea for libvirt (or any other user) to parse > QEMU source (whether it's C code or yaml) or other way around for users > to influence QEMU internals. NB It isn't for libvirt to parse at runtime, rather it is for libvirt maintainers to consume during dev, so libvirt keeps in sync with QEMU features. With regards, Daniel
On Fri, 8 Sep 2023 17:55:12 +0100 Daniel P. Berrangé <berrange@redhat.com> wrote: > On Fri, Sep 08, 2023 at 04:48:46PM +0200, Igor Mammedov wrote: > > On Fri, 8 Sep 2023 14:45:24 +0200 > > Tim Wiederhake <twiederh@redhat.com> wrote: > > > > > Synchronizing the list of cpu features and models with qemu is a recurring > > > task in libvirt. For x86, this is done by reading qom-list-properties for > > > max-x86_64-cpu and manually filtering out everthing that does not look like > > > a feature name, as well as parsing target/i386/cpu.c for cpu models. > > > > modulo fixing typos/name conflicts in 1st 3 patches, > > > > I don't think it's a great idea for libvirt (or any other user) to parse > > QEMU source (whether it's C code or yaml) or other way around for users > > to influence QEMU internals. > > NB It isn't for libvirt to parse at runtime, rather it is for libvirt > maintainers to consume during dev, so libvirt keeps in sync with QEMU > features. As QEMU dev, I'm not fond of code generators as sometimes they make difficult for me to read, and on to of that inventing new 'language' to describe features that works on some cases only (not everything is described in feature array, and for non x86 properties mostly coded in initfn/realizefn). (I'd dislike it less if it were part of QMP schema as it gets us closer to processing '-device' with QMP machinery). why not use existing QMP interface there as well (or alter it if it's not sufficient)? > > > With regards, > Daniel
On Mon, 2023-09-11 at 13:26 +0200, Igor Mammedov wrote: > On Fri, 8 Sep 2023 17:55:12 +0100 > Daniel P. Berrangé <berrange@redhat.com> wrote: > > > On Fri, Sep 08, 2023 at 04:48:46PM +0200, Igor Mammedov wrote: > > > On Fri, 8 Sep 2023 14:45:24 +0200 > > > Tim Wiederhake <twiederh@redhat.com> wrote: > > > > > > > Synchronizing the list of cpu features and models with qemu is > > > > a recurring > > > > task in libvirt. For x86, this is done by reading qom-list- > > > > properties for > > > > max-x86_64-cpu and manually filtering out everthing that does > > > > not look like > > > > a feature name, as well as parsing target/i386/cpu.c for cpu > > > > models. > > > > > > modulo fixing typos/name conflicts in 1st 3 patches, > > > > > > I don't think it's a great idea for libvirt (or any other user) > > > to parse > > > QEMU source (whether it's C code or yaml) or other way around for > > > users > > > to influence QEMU internals. > > > > NB It isn't for libvirt to parse at runtime, rather it is for > > libvirt > > maintainers to consume during dev, so libvirt keeps in sync with > > QEMU > > features. > > As QEMU dev, I'm not fond of code generators as sometimes they make > difficult for me to read, and on to of that inventing new 'language' > to describe features that works on some cases only (not everything > is described in feature array, and for non x86 properties mostly > coded in initfn/realizefn). > (I'd dislike it less if it were part of QMP schema as it gets us > closer to processing '-device' with QMP machinery). > > why not use existing QMP interface there as well (or alter it if > it's not sufficient)? > > I understand your concern regarding the legibility of generated code. If you have a look at patches 6 - 9, you will see that the only changes introduced are white space changes, harmonizing the usage of trailing commas, breaking lines, and filling the "feat_names" field fully. That, if anything, makes the code more readable in my opinion. The format of the yaml file is chosen to mimic struct FeatureWordInfo as closely as possible, it is effectively a 1:1 mapping. Regarding using the QMP interface: That is what libvirt is currently doing, see [1] if you are interested. The problem with that approach is that the response to { "execute": "qom-list-properties", "arguments": { "typename": "max-x86_64-cpu"} } does not distinguish between cpu features and other information about that cpu model, thus forcing libvirt to maintain a list of non- features. Examples of such items are "check", "enforce", "start- powered-off", or "migratable". Additionally, the response does not include information about the cpuid leaf, register and bit, or msr register respectively. This has to be supplemented manually. If there is a way to query for this information that I am not aware of, please let me know. Ultimately I would love to see libvirt stop querying qemu for x86 cpu feature information but instead have both qemu and libvirt query a common source / database for this kind of information, for all architectures, not only x86. Regards, Tim [1] https://gitlab.com/libvirt/libvirt/-/blob/master/src/cpu_map/sync_qemu_features_i386.py