| Message ID | 20220401224104.145961-5-victortoso@redhat.com |
|---|---|
| State | New |
| Headers | show |
| Series | qapi: add generator for Golang interface | expand |
On Sat, Apr 02, 2022 at 12:41:00AM +0200, Victor Toso wrote: > This patch handles QAPI union types and generates the equivalent data > structures and methods in Go to handle it. > > At the moment of this writing, it generates 67 structures. > > The QAPI union type can be summarized by its common members that are > defined in a @base struct and a @value. The @value type can vary and > depends on @base's field that we call @discriminator. The > @discriminator is always a Enum type. > > Golang does not have Unions. The generation of QAPI union type in Go > with this patch, follows similar approach to what is done for QAPI > struct types and QAPI alternate types. The common way to approach unions in Go is to just use a struct where each union case is an optional field, and declare that only one field must ever be set. ie type SocketAddressLegacy struct { // Value based on @type, possible types: Inet *InetSocketAddressWrapper Unix *UnixSocketAddressWrapper VSock *VsockSocketAddressWrapper FD *StringWrapper } When deserializing from JSON we populate exactly one of the optional fields. When serializing to JSON process the first field that is non-nil. Note, you don't actually need to include the discriminator as a field at all, since it is implicitly determined by whichever case is non-nil. Introducing the discriminator as a field just provides the possibility for the programmer to make inconsistent settings, for no gain. With regards, Daniel
Hi, On Tue, May 10, 2022 at 11:26:56AM +0100, Daniel P. Berrangé wrote: > On Sat, Apr 02, 2022 at 12:41:00AM +0200, Victor Toso wrote: > > This patch handles QAPI union types and generates the equivalent data > > structures and methods in Go to handle it. > > > > At the moment of this writing, it generates 67 structures. > > > > The QAPI union type can be summarized by its common members that are > > defined in a @base struct and a @value. The @value type can vary and > > depends on @base's field that we call @discriminator. The > > @discriminator is always a Enum type. > > > > Golang does not have Unions. The generation of QAPI union type in Go > > with this patch, follows similar approach to what is done for QAPI > > struct types and QAPI alternate types. > > The common way to approach unions in Go is to just use a struct > where each union case is an optional field, and declare that > only one field must ever be set. ie > > type SocketAddressLegacy struct { > // Value based on @type, possible types: > Inet *InetSocketAddressWrapper > Unix *UnixSocketAddressWrapper > VSock *VsockSocketAddressWrapper > FD *StringWrapper > } Like Alternates, I like this better. > When deserializing from JSON we populate exactly one of the > optional fields. > > When serializing to JSON process the first field that is > non-nil. > > Note, you don't actually need to include the discriminator as a > field at all, since it is implicitly determined by whichever > case is non-nil. Introducing the discriminator as a field just > provides the possibility for the programmer to make > inconsistent settings, for no gain. Sounds reasonable. We still need to implement Marshal/Unmarshal for unknow types (e.g: a new Type for SocketAddressLegacy was introduced in 7.1 and we should be able to know that current qapi-go version can't understand it). Cheers, Victor
On Tue, May 10, 2022 at 01:32:08PM +0200, Victor Toso wrote: > Hi, > > On Tue, May 10, 2022 at 11:26:56AM +0100, Daniel P. Berrangé wrote: > > On Sat, Apr 02, 2022 at 12:41:00AM +0200, Victor Toso wrote: > > > This patch handles QAPI union types and generates the equivalent data > > > structures and methods in Go to handle it. > > > > > > At the moment of this writing, it generates 67 structures. > > > > > > The QAPI union type can be summarized by its common members that are > > > defined in a @base struct and a @value. The @value type can vary and > > > depends on @base's field that we call @discriminator. The > > > @discriminator is always a Enum type. > > > > > > Golang does not have Unions. The generation of QAPI union type in Go > > > with this patch, follows similar approach to what is done for QAPI > > > struct types and QAPI alternate types. > > > > The common way to approach unions in Go is to just use a struct > > where each union case is an optional field, and declare that > > only one field must ever be set. ie > > > > type SocketAddressLegacy struct { > > // Value based on @type, possible types: > > Inet *InetSocketAddressWrapper > > Unix *UnixSocketAddressWrapper > > VSock *VsockSocketAddressWrapper > > FD *StringWrapper > > } > > Like Alternates, I like this better. > > > When deserializing from JSON we populate exactly one of the > > optional fields. > > > > When serializing to JSON process the first field that is > > non-nil. > > > > Note, you don't actually need to include the discriminator as a > > field at all, since it is implicitly determined by whichever > > case is non-nil. Introducing the discriminator as a field just > > provides the possibility for the programmer to make > > inconsistent settings, for no gain. > > Sounds reasonable. We still need to implement Marshal/Unmarshal > for unknow types (e.g: a new Type for SocketAddressLegacy was > introduced in 7.1 and we should be able to know that current > qapi-go version can't understand it). If there's a new type seen on the wire then, the easy option is to just not deserialize it at all. Return a SocketAddressLegacy struct with all fields nil, or perhaps return a full 'error', since this is likely to be significant functionalproblem for the application. With regards, Daniel
diff --git a/scripts/qapi/golang.py b/scripts/qapi/golang.py index 50e39f8925..0a1bf430ba 100644 --- a/scripts/qapi/golang.py +++ b/scripts/qapi/golang.py @@ -31,7 +31,7 @@ class QAPISchemaGenGolangVisitor(QAPISchemaVisitor): def __init__(self, prefix: str): super().__init__() - self.target = {name: "" for name in ["alternate", "enum", "helper", "struct"]} + self.target = {name: "" for name in ["alternate", "enum", "helper", "struct", "union"]} self.objects_seen = {} self.schema = None self._docmap = {} @@ -82,10 +82,10 @@ def visit_object_type(self: QAPISchemaGenGolangVisitor, members: List[QAPISchemaObjectTypeMember], variants: Optional[QAPISchemaVariants] ) -> None: - # Do not handle anything besides structs + # Do not handle anything besides struct and unions. if (name == self.schema.the_empty_object_type.name or not isinstance(name, str) or - info.defn_meta not in ["struct"]): + info.defn_meta not in ["struct", "union"]): return assert name not in self.objects_seen @@ -351,6 +351,93 @@ def generate_marshal_methods_enum(members: List[QAPISchemaEnumMember]) -> str: }} ''' +# Marshal methods for Union types +def generate_marshal_methods(type: str, + type_dict: Dict[str, str], + discriminator: str = "", + base: str = "") -> str: + assert base != "" + discriminator = "base." + discriminator + + switch_case_format = ''' + case {name}: + value := {case_type}{{}} + if err := json.Unmarshal(data, &value); err != nil {{ + return err + }} + s.Value = value''' + + if_supported_types = "" + added = {} + switch_cases = "" + for name in sorted(type_dict): + case_type = type_dict[name] + isptr = "*" if case_type[0] not in "*[" else "" + switch_cases += switch_case_format.format(name = name, + case_type = case_type) + if case_type not in added: + if_supported_types += f'''typestr != "{case_type}" &&\n\t\t''' + added[case_type] = True + + marshalfn = f''' +func (s {type}) MarshalJSON() ([]byte, error) {{ + base, err := json.Marshal(s.{base}) + if err != nil {{ + return nil, err + }} + + typestr := fmt.Sprintf("%T", s.Value) + typestr = typestr[strings.LastIndex(typestr, ".")+1:] + + // "The branches need not cover all possible enum values" + // This means that on Marshal, we can safely ignore empty values + if typestr == "<nil>" {{ + return []byte(base), nil + }} + + // Runtime check for supported value types + if {if_supported_types[:-6]} {{ + return nil, errors.New(fmt.Sprintf("Type is not supported: %s", typestr)) + }} + value, err := json.Marshal(s.Value) + if err != nil {{ + return nil, err + }} + + // Workaround to avoid checking s.Value being empty + if string(value) == "{{}}" {{ + return []byte(base), nil + }} + + // Removes the last '}}' from base and the first '{{' from value, in order to + // return a single JSON object. + result := fmt.Sprintf("%s,%s", base[:len(base)-1], value[1:]) + return []byte(result), nil +}} +''' + unmarshal_base = f''' + var base {base} + if err := json.Unmarshal(data, &base); err != nil {{ + return err + }} + s.{base} = base +''' + unmarshal_default_warn = f''' + default: + fmt.Println("Failed to decode {type}", {discriminator})''' + + return f'''{marshalfn} +func (s *{type}) UnmarshalJSON(data []byte) error {{ + {unmarshal_base} + switch {discriminator} {{ +{switch_cases[1:]} + {unmarshal_default_warn} + }} + + return nil +}} +''' + # Takes the documentation object of a specific type and returns # that type's documentation followed by its member's docs. def qapi_to_golang_struct_docs(doc: QAPIDoc) -> (str, Dict[str, str]): @@ -412,10 +499,37 @@ def qapi_to_golang_struct(name: str, field_doc = doc_fields.get(memb.name, "") own_fields += f"\t{field} {isptr}{member_type}{fieldtag}{field_doc}\n" + union_types = {} + variant_fields = "" + if variants: + variant_fields = f"// Value based on @{variants.tag_member.name}, possible types:" + for var in variants.variants: + if var.type.is_implicit(): + continue + + name = variants.tag_member._type_name + var.name.title().replace("-", "") + union_types[name] = var.type.name + variant_fields += f"\n\t// * {var.type.c_unboxed_type()}" + + variant_fields += f"\n\tValue Any" + all_fields = base_fields if len(base_fields) > 0 else "" all_fields += own_fields[:-1] if len(own_fields) > 0 else "" - - return generate_struct_type(type_name, all_fields, doc_struct) + all_fields += variant_fields if len(variant_fields) > 0 else "" + + unmarshal_fn = "" + if info.defn_meta == "union" and variants is not None: + # Union's without variants are the Union's base data structure. + # e.g: SchemaInfo's base is SchemainfoBase. + discriminator = qapi_to_field_name(variants.tag_member.name) + base = qapi_to_go_type_name(variants.tag_member.defined_in, + variants.tag_member.info.defn_meta) + unmarshal_fn = generate_marshal_methods(type_name, + union_types, + discriminator = discriminator, + base = base_type_name) + + return generate_struct_type(type_name, all_fields, doc_struct) + unmarshal_fn def qapi_schema_type_to_go_type(type: str) -> str: schema_types_to_go = {'str': 'string', 'null': 'nil', 'bool': 'bool',
This patch handles QAPI union types and generates the equivalent data structures and methods in Go to handle it. At the moment of this writing, it generates 67 structures. The QAPI union type can be summarized by its common members that are defined in a @base struct and a @value. The @value type can vary and depends on @base's field that we call @discriminator. The @discriminator is always a Enum type. Golang does not have Unions. The generation of QAPI union type in Go with this patch, follows similar approach to what is done for QAPI struct types and QAPI alternate types. Similarly to Go implementation of QAPI alternate types, we will implement the Marshaler and Unmarshaler interfaces to seamless decode from JSON objects to Golang structs and vice versa. Similarly to Go implementation of QAPI struct types, we will need to tag @base fields accordingly. The embedded documentation in Golang's structures and fields are particularly important here, to help developers know what Types to use for @value. Runtime checks too. Signed-off-by: Victor Toso <victortoso@redhat.com> --- scripts/qapi/golang.py | 124 +++++++++++++++++++++++++++++++++++++++-- 1 file changed, 119 insertions(+), 5 deletions(-)