| Message ID | 20171002053819.41878-1-aik@ozlabs.ru |
|---|---|
| State | Superseded |
| Headers | show |
| Series | [v2] fdt: Pass the resulting device tree to QEMU | expand |
On Mon, 2 Oct 2017 16:38:19 +1100 Alexey Kardashevskiy <aik@ozlabs.ru> wrote: > This creates flatten device tree and passes it to QEMU via a custom > hypercall right before jumping to RTAS. > > On a machine with 256 CPUs and 256 virtual Intel E1000 devices the blob > is 360KB (356KB structs and 20KB of strings), building such a tree takes > ~2s on a POWER8 box. A simple tree with 1 CPU and a couple of devices > takes 38ms and creates 16KB blob. > > This preloads strings with 40 property names from CPU and PCI device nodes > and the strings lookup only searches within these. Without string reusing > at all, the strings blob is 200KB and rendering time is 1.7sec; with > unlimited reusing, the strings blob is 4KB and rendering time is 2.8sec. > > Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru> > --- > > Changes: > v2: > * fixed comments from review > * added strings cache > * changed last_compat_vers from 0x17 to 0x16 as suggested by dwg > > --- > > I tested the blob by storing it from QEMU to a file and decompiling it; > this produces error which I do not really > understand as the name of the root is an empty string (literaly: > 00 00 00 01 00 00 00 00) and yet this error: > > aik@fstn1-p1:~$ dtc -f -I dtb -O dts -o dbg.dts dbg.dtb > ERROR (name_properties): "name" property in / is incorrect ("/" instead of base node name) > Warning: Input tree has errors, output forced > --- > lib/libhvcall/libhvcall.h | 1 + > board-qemu/slof/fdt.fs | 297 +++++++++++++++++++++++++++++++++++++++++++++- > board-qemu/slof/rtas.fs | 4 + > lib/libhvcall/hvcall.code | 5 + > lib/libhvcall/hvcall.in | 1 + > 5 files changed, 307 insertions(+), 1 deletion(-) > > diff --git a/lib/libhvcall/libhvcall.h b/lib/libhvcall/libhvcall.h > index 5776a2b..1c2d31b 100644 > --- a/lib/libhvcall/libhvcall.h > +++ b/lib/libhvcall/libhvcall.h > @@ -26,6 +26,7 @@ > #define KVMPPC_H_CAS (KVMPPC_HCALL_BASE + 0x2) > #define KVMPPC_H_RTAS_UPDATE (KVMPPC_HCALL_BASE + 0x3) > #define KVMPPC_H_UPDATE_PHANDLE (KVMPPC_HCALL_BASE + 0x4) Since KVMPPC_H_UPDATE_PHANDLE hasn't any user obviously, and is supposed to be obsoleted by this patch, maybe you can revert it in the first place, no ? > +#define KVMPPC_H_UPDATE_DT (KVMPPC_HCALL_BASE + 0x5) > > #ifndef __ASSEMBLY__ > > diff --git a/board-qemu/slof/fdt.fs b/board-qemu/slof/fdt.fs > index a24e344..ced9db3 100644 > --- a/board-qemu/slof/fdt.fs > +++ b/board-qemu/slof/fdt.fs > @@ -27,7 +27,7 @@ struct > 4 field >fdth_boot_cpu > 4 field >fdth_string_size > 4 field >fdth_struct_size > -drop > +constant /fdth > > h# d00dfeed constant OF_DT_HEADER > h# 1 constant OF_DT_BEGIN_NODE > @@ -449,4 +449,299 @@ r> drop > fdt-cas-fix? > ; > > +VARIABLE fdt-struct > +VARIABLE fdt-struct-cur > +VARIABLE fdt-strings > +VARIABLE fdt-strings-cache > +VARIABLE fdt-strings-cur > +VARIABLE fdt-strings-reused > +VARIABLE fdt-ms > + > +\ Adds data to struct blob, also adds terminating null if name is added > +: fdt-struct-add ( bytes len nullterminate -- ) > + >r > + dup >r ( bytes len r: nullterminate len ) > + fdt-struct-cur @ swap ( bytes cur len r: nullterminate len ) > + \ copy bytes to structs > + move > + fdt-struct-cur @ ( cur r: nullterminate len ) > + \ increment fdt-struct-cur > + r> + ( cur r: nullterminate ) > + \ add null if it is a string > + r> IF > + 0 over c! > + 1+ > + THEN > + \ and align to 4 bytes > + 3 + -4 and > + fdt-struct-cur ! > +; > + > +: fdt-encode-int ( token -- ) > + fdt-struct-cur @ l! > + /l fdt-struct-cur +! > +; > + > +: fdt-skip-string ( cur -- cur ) > + BEGIN > + dup c@ > + WHILE > + 1+ > + REPEAT > + 4 + -4 and > +; > + > +: zstring= ( str len zstr -- flag ) > + 2dup + c@ 0<> IF > + 3drop false > + EXIT > + THEN > + swap comp 0= > +; > + > +: fdt-find-string ( name namelen -- nameoff true | false ) > + fdt-strings @ > + BEGIN > + dup fdt-strings-cache @ < > + WHILE > + 3dup zstring= IF > + fdt-strings @ - > + -rot > + 2drop > + true > + EXIT > + THEN > + fdt-skip-string > + REPEAT > + 3drop > + false > +; > + > +: fdt-add-string ( name namelen -- nameoff ) > + fdt-strings-cur @ > + fdt-strings @ - >r ( name namelen r: nameoff ) > + dup >r ( name namelen r: nameoff namelen ) > + > + \ copy string > + fdt-strings-cur @ swap ( name cur namelen r: nameoff namelen ) > + move ( r: nameoff namelen ) > + > + \ increment cur and null terminate > + fdt-strings-cur @ > + r> + ( cur r: nameoff ) > + \ add null if it is a string > + 0 over c! > + \ add 1 byte for the null and align to 4 bytes > + 4 + -4 and > + fdt-strings-cur ! ( r: nameoff ) > + r> ( nameoff ) > +; > + > +: fdt-get-string ( name namelen -- nameoff ) > + 2dup fdt-find-string IF > + -rot 2drop > + fdt-debug IF > + 1 fdt-strings-reused +! > + THEN > + EXIT > + THEN > + fdt-add-string > +; > + > +: fdt-begin-node ( name namelen -- ) > + OF_DT_BEGIN_NODE fdt-encode-int > + 2dup 1 = swap c@ [char] / = and \ is it "/"? > + IF > + 2drop s" " \ dtc is still unhappy though > + THEN > + true fdt-struct-add > +; > + > +: fdt-end-node ( -- ) > + OF_DT_END_NODE fdt-encode-int > +; > + > +: fdt-prop ( prop len name namelen -- ) > + OF_DT_PROP fdt-encode-int > + > + \ get string offset > + fdt-get-string ( prop len nameoff ) > + > + \ store len and nameoff > + over fdt-encode-int > + fdt-encode-int ( prop len ) > + > + \ now add the bytes > + false fdt-struct-add > +; > + > +: fdt-end ( -- ) > + OF_DT_END fdt-encode-int > +; > + > +: fdt-properties ( phandle -- ) > + dup encode-int s" phandle" fdt-prop > + >r > + s" " > + BEGIN > + r@ next-property > + WHILE > + 2dup > + 2dup r@ get-property > + not IF > + 2swap fdt-prop > + THEN > + REPEAT > + r> > + drop > +; > + > +: fdt-flatten-node ( node -- ) > +\ fdt-debug IF dup node>path type cr THEN > + dup node>qname fdt-begin-node > + dup fdt-properties > + child > + BEGIN > + dup > + WHILE > + dup recurse > + peer > + REPEAT > + drop > + fdt-end-node > +; > + > +: fdt-strings-preload ( -- ) > + s" reg" fdt-add-string drop > + s" status" fdt-add-string drop > + s" 64-bit" fdt-add-string drop > + s" phandle" fdt-add-string drop > + s" ibm,vmx" fdt-add-string drop > + s" ibm,dfp" fdt-add-string drop > + s" slb-size" fdt-add-string drop > + s" ibm,purr" fdt-add-string drop > + s" vendor-id" fdt-add-string drop > + s" device-id" fdt-add-string drop > + s" min-grant" fdt-add-string drop > + s" class-code" fdt-add-string drop > + s" compatible" fdt-add-string drop > + s" interrupts" fdt-add-string drop > + s" cpu-version" fdt-add-string drop > + s" #size-cells" fdt-add-string drop > + s" ibm,req#msi" fdt-add-string drop > + s" revision-id" fdt-add-string drop > + s" device_type" fdt-add-string drop > + s" max-latency" fdt-add-string drop > + s" ibm,chip-id" fdt-add-string drop > + s" ibm,pft-size" fdt-add-string drop > + s" ibm,slb-size" fdt-add-string drop > + s" devsel-speed" fdt-add-string drop > + s" ibm,loc-code" fdt-add-string drop > + s" subsystem-id" fdt-add-string drop > + s" d-cache-size" fdt-add-string drop > + s" i-cache-size" fdt-add-string drop > + s" #address-cells" fdt-add-string drop > + s" clock-frequency" fdt-add-string drop > + s" cache-line-size" fdt-add-string drop > + s" ibm,pa-features" fdt-add-string drop > + s" ibm,my-drc-index" fdt-add-string drop > + s" d-cache-line-size" fdt-add-string drop > + s" i-cache-line-size" fdt-add-string drop > + s" assigned-addresses" fdt-add-string drop > + s" d-cache-block-size" fdt-add-string drop > + s" i-cache-block-size" fdt-add-string drop > + s" timebase-frequency" fdt-add-string drop > + s" subsystem-vendor-id" fdt-add-string drop > + s" ibm,segment-page-sizes" fdt-add-string drop > + s" ibm,ppc-interrupt-server#s" fdt-add-string drop > + s" ibm,processor-segment-sizes" fdt-add-string drop > + s" ibm,ppc-interrupt-gserver#s" fdt-add-string drop > +; > + > +: fdt-append-blob ( bytes cur blob -- cur ) > + 3dup -rot swap move > + drop + > +; > + > +: fdt-flatten-tree ( root -- tree ) > + 200000 alloc-mem dup fdt-struct-cur ! fdt-struct ! > + 200000 alloc-mem dup fdt-strings-cur ! fdt-strings ! > + > + fdt-debug IF > + 0 fdt-strings-reused ! > + milliseconds fdt-ms ! > + THEN > + > + \ Preload strings cache > + fdt-strings-preload > + fdt-strings-cur @ fdt-strings-cache ! > + \ Render the blobs > + fdt-flatten-node > + fdt-end > + > + \ Calculate strings and struct sizes > + fdt-struct-cur @ fdt-struct @ - > + fdt-strings-cur @ fdt-strings @ - ( struct-len strings-len ) > + > + 2dup + /fdth + > + 10 + \ Reserve 16 bytes and an empty reserved block > + > + fdt-debug IF > + 3dup > + ." FDT flat size=" .d cr > + ." Strings size=" .d cr > + ." Struct size=" .d cr > + ." Reused strings=" fdt-strings-reused @ .d cr > + milliseconds fdt-ms @ - > + ." Took " .d ." ms" cr > + THEN > + > + \ Allocate flatten DT blob > + dup alloc-mem ( struct-len strings-len total-len fdt ) > + >r ( struct-len strings-len total-len r: fdt ) > + > + \ Write header > + OF_DT_HEADER r@ >fdth_magic l! > + dup r@ >fdth_tsize l! > + /fdth 10 + 2 pick + r@ >fdth_struct_off l! > + /fdth 10 + r@ >fdth_string_off l! > + /fdth r@ >fdth_rsvmap_off l! > + 17 r@ >fdth_version l! > + 16 r@ >fdth_compat_vers l! > + 0 r@ >fdth_boot_cpu l! > + over r@ >fdth_string_size l! > + 2 pick r@ >fdth_struct_size l! > + ( struct-len strings-len total-len r: fdt ) > + > + drop ( struct-len strings-len r: fdt ) > + r@ /fdth + ( struct-len strings-len cur r: fdt ) > + > + \ Write the reserved entry > + 0 over ! > + cell+ > + 0 over ! > + cell+ ( struct-len strings-len cur r: fdt ) > + > + \ Write strings and struct blobs > + fdt-strings @ fdt-append-blob > + fdt-struct @ fdt-append-blob > + drop > + > + \ Free temporary blobs > + fdt-struct @ 200000 free-mem > + fdt-strings @ 200000 free-mem > + > + \ Return fdt > + r> > +; > + > +: fdt-flatten-tree-free ( tree ) > + dup >fdth_tsize l@ free-mem > +; > + > +: fdt ( -- ) > + " /" find-node > + fdt-flatten-tree > +; > + > s" /" find-node fdt-fix-phandles > diff --git a/board-qemu/slof/rtas.fs b/board-qemu/slof/rtas.fs > index 54d3929..5beb079 100644 > --- a/board-qemu/slof/rtas.fs > +++ b/board-qemu/slof/rtas.fs > @@ -98,6 +98,10 @@ find-qemu-rtas > ; > > : rtas-quiesce ( -- ) > + " /" find-node > + fdt-flatten-tree > + dup hv-update-dt > + fdt-flatten-tree-free > " quiesce" rtas-get-token rtas-cb rtas>token l! > 0 rtas-cb rtas>nargs l! > 0 rtas-cb rtas>nret l! > diff --git a/lib/libhvcall/hvcall.code b/lib/libhvcall/hvcall.code > index 8349748..6ff5715 100644 > --- a/lib/libhvcall/hvcall.code > +++ b/lib/libhvcall/hvcall.code > @@ -136,3 +136,8 @@ PRIM(hv_X2d_update_X2d_phandle) > uint32_t old_phandle = TOS.u; > TOS.u = hv_generic(KVMPPC_H_UPDATE_PHANDLE, old_phandle, new_phandle); > MIRP > + > +PRIM(hv_X2d_update_X2d_dt) > + unsigned long dt = TOS.u; > + TOS.u = hv_generic(KVMPPC_H_UPDATE_DT, dt); > +MIRP > diff --git a/lib/libhvcall/hvcall.in b/lib/libhvcall/hvcall.in > index ab7513a..b59e3f7 100644 > --- a/lib/libhvcall/hvcall.in > +++ b/lib/libhvcall/hvcall.in > @@ -32,4 +32,5 @@ cod(hv-logical-memop) > cod(hv-cas) > cod(hv-rtas-update) > cod(hv-update-phandle) > +cod(hv-update-dt) > cod(get-print-version)
On 02/10/17 18:47, Greg Kurz wrote: > On Mon, 2 Oct 2017 16:38:19 +1100 > Alexey Kardashevskiy <aik@ozlabs.ru> wrote: > >> This creates flatten device tree and passes it to QEMU via a custom >> hypercall right before jumping to RTAS. >> >> On a machine with 256 CPUs and 256 virtual Intel E1000 devices the blob >> is 360KB (356KB structs and 20KB of strings), building such a tree takes >> ~2s on a POWER8 box. A simple tree with 1 CPU and a couple of devices >> takes 38ms and creates 16KB blob. >> >> This preloads strings with 40 property names from CPU and PCI device nodes >> and the strings lookup only searches within these. Without string reusing >> at all, the strings blob is 200KB and rendering time is 1.7sec; with >> unlimited reusing, the strings blob is 4KB and rendering time is 2.8sec. >> >> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru> >> --- >> >> Changes: >> v2: >> * fixed comments from review >> * added strings cache >> * changed last_compat_vers from 0x17 to 0x16 as suggested by dwg >> >> --- >> >> I tested the blob by storing it from QEMU to a file and decompiling it; >> this produces error which I do not really >> understand as the name of the root is an empty string (literaly: >> 00 00 00 01 00 00 00 00) and yet this error: >> >> aik@fstn1-p1:~$ dtc -f -I dtb -O dts -o dbg.dts dbg.dtb >> ERROR (name_properties): "name" property in / is incorrect ("/" instead of base node name) >> Warning: Input tree has errors, output forced >> --- >> lib/libhvcall/libhvcall.h | 1 + >> board-qemu/slof/fdt.fs | 297 +++++++++++++++++++++++++++++++++++++++++++++- >> board-qemu/slof/rtas.fs | 4 + >> lib/libhvcall/hvcall.code | 5 + >> lib/libhvcall/hvcall.in | 1 + >> 5 files changed, 307 insertions(+), 1 deletion(-) >> >> diff --git a/lib/libhvcall/libhvcall.h b/lib/libhvcall/libhvcall.h >> index 5776a2b..1c2d31b 100644 >> --- a/lib/libhvcall/libhvcall.h >> +++ b/lib/libhvcall/libhvcall.h >> @@ -26,6 +26,7 @@ >> #define KVMPPC_H_CAS (KVMPPC_HCALL_BASE + 0x2) >> #define KVMPPC_H_RTAS_UPDATE (KVMPPC_HCALL_BASE + 0x3) >> #define KVMPPC_H_UPDATE_PHANDLE (KVMPPC_HCALL_BASE + 0x4) > > Since KVMPPC_H_UPDATE_PHANDLE hasn't any user obviously, and is > supposed to be obsoleted by this patch, maybe you can revert it > in the first place, no ? Sure, when/if this one is ready to go to upstream and QEMU at the time won't have H_UPDATE_PHANDLE pulled :)
On Mon, 2 Oct 2017 19:26:41 +1100 Alexey Kardashevskiy <aik@ozlabs.ru> wrote: > On 02/10/17 18:47, Greg Kurz wrote: > > On Mon, 2 Oct 2017 16:38:19 +1100 > > Alexey Kardashevskiy <aik@ozlabs.ru> wrote: > > > >> This creates flatten device tree and passes it to QEMU via a custom > >> hypercall right before jumping to RTAS. > >> > >> On a machine with 256 CPUs and 256 virtual Intel E1000 devices the blob > >> is 360KB (356KB structs and 20KB of strings), building such a tree takes > >> ~2s on a POWER8 box. A simple tree with 1 CPU and a couple of devices > >> takes 38ms and creates 16KB blob. > >> > >> This preloads strings with 40 property names from CPU and PCI device nodes > >> and the strings lookup only searches within these. Without string reusing > >> at all, the strings blob is 200KB and rendering time is 1.7sec; with > >> unlimited reusing, the strings blob is 4KB and rendering time is 2.8sec. > >> > >> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru> > >> --- > >> > >> Changes: > >> v2: > >> * fixed comments from review > >> * added strings cache > >> * changed last_compat_vers from 0x17 to 0x16 as suggested by dwg > >> > >> --- > >> > >> I tested the blob by storing it from QEMU to a file and decompiling it; > >> this produces error which I do not really > >> understand as the name of the root is an empty string (literaly: > >> 00 00 00 01 00 00 00 00) and yet this error: > >> > >> aik@fstn1-p1:~$ dtc -f -I dtb -O dts -o dbg.dts dbg.dtb > >> ERROR (name_properties): "name" property in / is incorrect ("/" instead of base node name) > >> Warning: Input tree has errors, output forced > >> --- > >> lib/libhvcall/libhvcall.h | 1 + > >> board-qemu/slof/fdt.fs | 297 +++++++++++++++++++++++++++++++++++++++++++++- > >> board-qemu/slof/rtas.fs | 4 + > >> lib/libhvcall/hvcall.code | 5 + > >> lib/libhvcall/hvcall.in | 1 + > >> 5 files changed, 307 insertions(+), 1 deletion(-) > >> > >> diff --git a/lib/libhvcall/libhvcall.h b/lib/libhvcall/libhvcall.h > >> index 5776a2b..1c2d31b 100644 > >> --- a/lib/libhvcall/libhvcall.h > >> +++ b/lib/libhvcall/libhvcall.h > >> @@ -26,6 +26,7 @@ > >> #define KVMPPC_H_CAS (KVMPPC_HCALL_BASE + 0x2) > >> #define KVMPPC_H_RTAS_UPDATE (KVMPPC_HCALL_BASE + 0x3) > >> #define KVMPPC_H_UPDATE_PHANDLE (KVMPPC_HCALL_BASE + 0x4) > > > > Since KVMPPC_H_UPDATE_PHANDLE hasn't any user obviously, and is > > supposed to be obsoleted by this patch, maybe you can revert it > > in the first place, no ? > > Sure, when/if this one is ready to go to upstream and QEMU at the time > won't have H_UPDATE_PHANDLE pulled :) > I hardly believe this may happen :)
On Mon, 2 Oct 2017 16:38:19 +1100 Alexey Kardashevskiy <aik@ozlabs.ru> wrote: > This creates flatten device tree and passes it to QEMU via a custom > hypercall right before jumping to RTAS. > > On a machine with 256 CPUs and 256 virtual Intel E1000 devices the blob > is 360KB (356KB structs and 20KB of strings), building such a tree takes > ~2s on a POWER8 box. A simple tree with 1 CPU and a couple of devices > takes 38ms and creates 16KB blob. > > This preloads strings with 40 property names from CPU and PCI device nodes > and the strings lookup only searches within these. Without string reusing > at all, the strings blob is 200KB and rendering time is 1.7sec; with > unlimited reusing, the strings blob is 4KB and rendering time is 2.8sec. > > Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru> > --- > > Changes: > v2: > * fixed comments from review > * added strings cache > * changed last_compat_vers from 0x17 to 0x16 as suggested by dwg > I wanted to give a try with PHB hotplug and SLOF breaks, because... [...] > diff --git a/board-qemu/slof/rtas.fs b/board-qemu/slof/rtas.fs > index 54d3929..5beb079 100644 > --- a/board-qemu/slof/rtas.fs > +++ b/board-qemu/slof/rtas.fs > @@ -98,6 +98,10 @@ find-qemu-rtas > ; > > : rtas-quiesce ( -- ) > + " /" find-node > + fdt-flatten-tree > + dup hv-update-dt .. this leaves the hcall return value on the stack. Maybe add: ?dup IF \ Ignore hcall not implemented error, print error otherwise dup -2 <> IF ." HV-UPDATE-DT error: " . cr ELSE drop THEN THEN > + fdt-flatten-tree-free > " quiesce" rtas-get-token rtas-cb rtas>token l! > 0 rtas-cb rtas>nargs l! > 0 rtas-cb rtas>nret l! > diff --git a/lib/libhvcall/hvcall.code b/lib/libhvcall/hvcall.code > index 8349748..6ff5715 100644 > --- a/lib/libhvcall/hvcall.code > +++ b/lib/libhvcall/hvcall.code > @@ -136,3 +136,8 @@ PRIM(hv_X2d_update_X2d_phandle) > uint32_t old_phandle = TOS.u; > TOS.u = hv_generic(KVMPPC_H_UPDATE_PHANDLE, old_phandle, new_phandle); > MIRP > + > +PRIM(hv_X2d_update_X2d_dt) > + unsigned long dt = TOS.u; > + TOS.u = hv_generic(KVMPPC_H_UPDATE_DT, dt); > +MIRP > diff --git a/lib/libhvcall/hvcall.in b/lib/libhvcall/hvcall.in > index ab7513a..b59e3f7 100644 > --- a/lib/libhvcall/hvcall.in > +++ b/lib/libhvcall/hvcall.in > @@ -32,4 +32,5 @@ cod(hv-logical-memop) > cod(hv-cas) > cod(hv-rtas-update) > cod(hv-update-phandle) > +cod(hv-update-dt) > cod(get-print-version)
On 03/10/17 03:18, Greg Kurz wrote: > On Mon, 2 Oct 2017 16:38:19 +1100 > Alexey Kardashevskiy <aik@ozlabs.ru> wrote: > >> This creates flatten device tree and passes it to QEMU via a custom >> hypercall right before jumping to RTAS. >> >> On a machine with 256 CPUs and 256 virtual Intel E1000 devices the blob >> is 360KB (356KB structs and 20KB of strings), building such a tree takes >> ~2s on a POWER8 box. A simple tree with 1 CPU and a couple of devices >> takes 38ms and creates 16KB blob. >> >> This preloads strings with 40 property names from CPU and PCI device nodes >> and the strings lookup only searches within these. Without string reusing >> at all, the strings blob is 200KB and rendering time is 1.7sec; with >> unlimited reusing, the strings blob is 4KB and rendering time is 2.8sec. >> >> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru> >> --- >> >> Changes: >> v2: >> * fixed comments from review >> * added strings cache >> * changed last_compat_vers from 0x17 to 0x16 as suggested by dwg >> > > I wanted to give a try with PHB hotplug and SLOF breaks, because... How does it break exactly? I agree I do not clear the stack but it did not show up in any form :-/ > > [...] >> diff --git a/board-qemu/slof/rtas.fs b/board-qemu/slof/rtas.fs >> index 54d3929..5beb079 100644 >> --- a/board-qemu/slof/rtas.fs >> +++ b/board-qemu/slof/rtas.fs >> @@ -98,6 +98,10 @@ find-qemu-rtas >> ; >> >> : rtas-quiesce ( -- ) >> + " /" find-node >> + fdt-flatten-tree >> + dup hv-update-dt > > .. this leaves the hcall return value on the stack. Maybe add: > > ?dup IF > \ Ignore hcall not implemented error, print error otherwise > dup -2 <> IF ." HV-UPDATE-DT error: " . cr ELSE drop THEN > THEN > >> + fdt-flatten-tree-free >> " quiesce" rtas-get-token rtas-cb rtas>token l! >> 0 rtas-cb rtas>nargs l! >> 0 rtas-cb rtas>nret l! >> diff --git a/lib/libhvcall/hvcall.code b/lib/libhvcall/hvcall.code >> index 8349748..6ff5715 100644 >> --- a/lib/libhvcall/hvcall.code >> +++ b/lib/libhvcall/hvcall.code >> @@ -136,3 +136,8 @@ PRIM(hv_X2d_update_X2d_phandle) >> uint32_t old_phandle = TOS.u; >> TOS.u = hv_generic(KVMPPC_H_UPDATE_PHANDLE, old_phandle, new_phandle); >> MIRP >> + >> +PRIM(hv_X2d_update_X2d_dt) >> + unsigned long dt = TOS.u; >> + TOS.u = hv_generic(KVMPPC_H_UPDATE_DT, dt); >> +MIRP >> diff --git a/lib/libhvcall/hvcall.in b/lib/libhvcall/hvcall.in >> index ab7513a..b59e3f7 100644 >> --- a/lib/libhvcall/hvcall.in >> +++ b/lib/libhvcall/hvcall.in >> @@ -32,4 +32,5 @@ cod(hv-logical-memop) >> cod(hv-cas) >> cod(hv-rtas-update) >> cod(hv-update-phandle) >> +cod(hv-update-dt) >> cod(get-print-version) >
On Tue, 3 Oct 2017 09:22:38 +1100 Alexey Kardashevskiy <aik@ozlabs.ru> wrote: > On 03/10/17 03:18, Greg Kurz wrote: > > On Mon, 2 Oct 2017 16:38:19 +1100 > > Alexey Kardashevskiy <aik@ozlabs.ru> wrote: > > > >> This creates flatten device tree and passes it to QEMU via a custom > >> hypercall right before jumping to RTAS. > >> > >> On a machine with 256 CPUs and 256 virtual Intel E1000 devices the blob > >> is 360KB (356KB structs and 20KB of strings), building such a tree takes > >> ~2s on a POWER8 box. A simple tree with 1 CPU and a couple of devices > >> takes 38ms and creates 16KB blob. > >> > >> This preloads strings with 40 property names from CPU and PCI device nodes > >> and the strings lookup only searches within these. Without string reusing > >> at all, the strings blob is 200KB and rendering time is 1.7sec; with > >> unlimited reusing, the strings blob is 4KB and rendering time is 2.8sec. > >> > >> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru> > >> --- > >> > >> Changes: > >> v2: > >> * fixed comments from review > >> * added strings cache > >> * changed last_compat_vers from 0x17 to 0x16 as suggested by dwg > >> > > > > I wanted to give a try with PHB hotplug and SLOF breaks, because... > > How does it break exactly? I agree I do not clear the stack but it did not > show up in any form :-/ > Device tree strings 0x0000000004c90000 -> 0x0000000004c90a88 Device tree struct 0x0000000004ca0000 -> 0x0000000004cb0000 +++Q+++ (15990) h_update_dt 1676: DT at 7e43b000 (7e43b000) 19340 bytes ( 700 ) Program Exception [ 7dc4b218 ] R0 .. R7 R8 .. R15 R16 .. R23 R24 .. R31 000000007dbe2ea8 000000007e463010 0000000000000000 0000000000000006 000000007e666fe0 000000007dc4b220 0000000000000000 000000007dc05500 000000007dc0eb00 000000007e711048 000000007e463010 000000007dc090e8 000000007dc4d000 000000007dbe8510 000000007dc092b8 0000000000000003 000000007dc5f320 000000007dbe84f0 0000000000008000 000000000000f001 000000007dc4b218 0000000000000000 000000000000f003 ffffffffffffffff 000000007e711050 0000000000000000 000000000000f004 000000007dc0ae08 0000000000000003 0000000000000000 000000000000f005 0000000000000000 CR / XER LR / CTR SRR0 / SRR1 DAR / DSISR 80000408 000000007dbe3534 000000007dc0ae08 0000000000000000 0000000000000000 000000007dc0ae08 8000000000081000 00000000 45 > ( 300 ) Data Storage Exception [ 7dc4b240 ] R0 .. R7 R8 .. R15 R16 .. R23 R24 .. R31 000000007dbe0308 000000007e463010 0000000000000000 0000000000000006 000000007e666fe0 4800000824000051 0000000000000000 000000007dc05500 000000007dc0eb00 000000007dc4b260 000000007e463010 000000007dc090e8 0000000000000064 000000007dc4eb00 000000007dc092b8 0000000000000003 0000000000000000 000000007dbe84f0 0000000000008000 000000000000f001 000000007dc4b240 0000000000000000 000000000000f003 ffffffffffffffff 000000007dc09f30 0000000000000000 000000000000f004 000000007dbe3ba0 4800000824000051 0000000000000000 000000000000f005 000000007dc099a8 CR / XER LR / CTR SRR0 / SRR1 DAR / DSISR 80000404 000000007dbe3490 000000007dbe42c4 4800000824000051 0000000020000000 000000007dbe3ba0 8000000000001000 40000000 I presume that this is the consequence of having 0 on top of the stack when calling fdt-flatten-tree-free, or am I missing something ? > > > > > [...] > >> diff --git a/board-qemu/slof/rtas.fs b/board-qemu/slof/rtas.fs > >> index 54d3929..5beb079 100644 > >> --- a/board-qemu/slof/rtas.fs > >> +++ b/board-qemu/slof/rtas.fs > >> @@ -98,6 +98,10 @@ find-qemu-rtas > >> ; > >> > >> : rtas-quiesce ( -- ) > >> + " /" find-node > >> + fdt-flatten-tree > >> + dup hv-update-dt > > > > .. this leaves the hcall return value on the stack. Maybe add: > > > > ?dup IF > > \ Ignore hcall not implemented error, print error otherwise > > dup -2 <> IF ." HV-UPDATE-DT error: " . cr ELSE drop THEN > > THEN > > > >> + fdt-flatten-tree-free > >> " quiesce" rtas-get-token rtas-cb rtas>token l! > >> 0 rtas-cb rtas>nargs l! > >> 0 rtas-cb rtas>nret l! > >> diff --git a/lib/libhvcall/hvcall.code b/lib/libhvcall/hvcall.code > >> index 8349748..6ff5715 100644 > >> --- a/lib/libhvcall/hvcall.code > >> +++ b/lib/libhvcall/hvcall.code > >> @@ -136,3 +136,8 @@ PRIM(hv_X2d_update_X2d_phandle) > >> uint32_t old_phandle = TOS.u; > >> TOS.u = hv_generic(KVMPPC_H_UPDATE_PHANDLE, old_phandle, new_phandle); > >> MIRP > >> + > >> +PRIM(hv_X2d_update_X2d_dt) > >> + unsigned long dt = TOS.u; > >> + TOS.u = hv_generic(KVMPPC_H_UPDATE_DT, dt); > >> +MIRP > >> diff --git a/lib/libhvcall/hvcall.in b/lib/libhvcall/hvcall.in > >> index ab7513a..b59e3f7 100644 > >> --- a/lib/libhvcall/hvcall.in > >> +++ b/lib/libhvcall/hvcall.in > >> @@ -32,4 +32,5 @@ cod(hv-logical-memop) > >> cod(hv-cas) > >> cod(hv-rtas-update) > >> cod(hv-update-phandle) > >> +cod(hv-update-dt) > >> cod(get-print-version) > > > >
On Mon, 2 Oct 2017 16:38:19 +1100 Alexey Kardashevskiy <aik@ozlabs.ru> wrote: > This creates flatten device tree and passes it to QEMU via a custom > hypercall right before jumping to RTAS. > > On a machine with 256 CPUs and 256 virtual Intel E1000 devices the blob > is 360KB (356KB structs and 20KB of strings), building such a tree takes > ~2s on a POWER8 box. A simple tree with 1 CPU and a couple of devices > takes 38ms and creates 16KB blob. > > This preloads strings with 40 property names from CPU and PCI device nodes > and the strings lookup only searches within these. Without string reusing > at all, the strings blob is 200KB and rendering time is 1.7sec; with > unlimited reusing, the strings blob is 4KB and rendering time is 2.8sec. > > Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru> > --- > > Changes: > v2: > * fixed comments from review > * added strings cache > * changed last_compat_vers from 0x17 to 0x16 as suggested by dwg > It isn't 0x17 and 0x16 but 0x11 and 0x10 actually, fdt-flatten-tree needs to be changed accordingly :) > --- > > I tested the blob by storing it from QEMU to a file and decompiling it; > this produces error which I do not really > understand as the name of the root is an empty string (literaly: > 00 00 00 01 00 00 00 00) and yet this error: > > aik@fstn1-p1:~$ dtc -f -I dtb -O dts -o dbg.dts dbg.dtb > ERROR (name_properties): "name" property in / is incorrect ("/" instead of base node name) > Warning: Input tree has errors, output forced > --- > lib/libhvcall/libhvcall.h | 1 + > board-qemu/slof/fdt.fs | 297 +++++++++++++++++++++++++++++++++++++++++++++- > board-qemu/slof/rtas.fs | 4 + > lib/libhvcall/hvcall.code | 5 + > lib/libhvcall/hvcall.in | 1 + > 5 files changed, 307 insertions(+), 1 deletion(-) > > diff --git a/lib/libhvcall/libhvcall.h b/lib/libhvcall/libhvcall.h > index 5776a2b..1c2d31b 100644 > --- a/lib/libhvcall/libhvcall.h > +++ b/lib/libhvcall/libhvcall.h > @@ -26,6 +26,7 @@ > #define KVMPPC_H_CAS (KVMPPC_HCALL_BASE + 0x2) > #define KVMPPC_H_RTAS_UPDATE (KVMPPC_HCALL_BASE + 0x3) > #define KVMPPC_H_UPDATE_PHANDLE (KVMPPC_HCALL_BASE + 0x4) > +#define KVMPPC_H_UPDATE_DT (KVMPPC_HCALL_BASE + 0x5) > > #ifndef __ASSEMBLY__ > > diff --git a/board-qemu/slof/fdt.fs b/board-qemu/slof/fdt.fs > index a24e344..ced9db3 100644 > --- a/board-qemu/slof/fdt.fs > +++ b/board-qemu/slof/fdt.fs > @@ -27,7 +27,7 @@ struct > 4 field >fdth_boot_cpu > 4 field >fdth_string_size > 4 field >fdth_struct_size > -drop > +constant /fdth > > h# d00dfeed constant OF_DT_HEADER > h# 1 constant OF_DT_BEGIN_NODE > @@ -449,4 +449,299 @@ r> drop > fdt-cas-fix? > ; > > +VARIABLE fdt-struct > +VARIABLE fdt-struct-cur > +VARIABLE fdt-strings > +VARIABLE fdt-strings-cache > +VARIABLE fdt-strings-cur > +VARIABLE fdt-strings-reused > +VARIABLE fdt-ms > + > +\ Adds data to struct blob, also adds terminating null if name is added > +: fdt-struct-add ( bytes len nullterminate -- ) > + >r > + dup >r ( bytes len r: nullterminate len ) > + fdt-struct-cur @ swap ( bytes cur len r: nullterminate len ) > + \ copy bytes to structs > + move > + fdt-struct-cur @ ( cur r: nullterminate len ) > + \ increment fdt-struct-cur > + r> + ( cur r: nullterminate ) > + \ add null if it is a string > + r> IF > + 0 over c! > + 1+ > + THEN > + \ and align to 4 bytes > + 3 + -4 and > + fdt-struct-cur ! > +; > + > +: fdt-encode-int ( token -- ) > + fdt-struct-cur @ l! > + /l fdt-struct-cur +! > +; > + > +: fdt-skip-string ( cur -- cur ) > + BEGIN > + dup c@ > + WHILE > + 1+ > + REPEAT > + 4 + -4 and > +; > + > +: zstring= ( str len zstr -- flag ) > + 2dup + c@ 0<> IF > + 3drop false > + EXIT > + THEN > + swap comp 0= > +; > + > +: fdt-find-string ( name namelen -- nameoff true | false ) > + fdt-strings @ > + BEGIN > + dup fdt-strings-cache @ < > + WHILE > + 3dup zstring= IF > + fdt-strings @ - > + -rot > + 2drop > + true > + EXIT > + THEN > + fdt-skip-string > + REPEAT > + 3drop > + false > +; > + > +: fdt-add-string ( name namelen -- nameoff ) > + fdt-strings-cur @ > + fdt-strings @ - >r ( name namelen r: nameoff ) > + dup >r ( name namelen r: nameoff namelen ) > + > + \ copy string > + fdt-strings-cur @ swap ( name cur namelen r: nameoff namelen ) > + move ( r: nameoff namelen ) > + > + \ increment cur and null terminate > + fdt-strings-cur @ > + r> + ( cur r: nameoff ) > + \ add null if it is a string > + 0 over c! > + \ add 1 byte for the null and align to 4 bytes > + 4 + -4 and > + fdt-strings-cur ! ( r: nameoff ) > + r> ( nameoff ) > +; > + > +: fdt-get-string ( name namelen -- nameoff ) > + 2dup fdt-find-string IF > + -rot 2drop > + fdt-debug IF > + 1 fdt-strings-reused +! > + THEN > + EXIT > + THEN > + fdt-add-string > +; > + > +: fdt-begin-node ( name namelen -- ) > + OF_DT_BEGIN_NODE fdt-encode-int > + 2dup 1 = swap c@ [char] / = and \ is it "/"? > + IF > + 2drop s" " \ dtc is still unhappy though > + THEN > + true fdt-struct-add > +; > + > +: fdt-end-node ( -- ) > + OF_DT_END_NODE fdt-encode-int > +; > + > +: fdt-prop ( prop len name namelen -- ) > + OF_DT_PROP fdt-encode-int > + > + \ get string offset > + fdt-get-string ( prop len nameoff ) > + > + \ store len and nameoff > + over fdt-encode-int > + fdt-encode-int ( prop len ) > + > + \ now add the bytes > + false fdt-struct-add > +; > + > +: fdt-end ( -- ) > + OF_DT_END fdt-encode-int > +; > + > +: fdt-properties ( phandle -- ) > + dup encode-int s" phandle" fdt-prop > + >r > + s" " > + BEGIN > + r@ next-property > + WHILE > + 2dup > + 2dup r@ get-property > + not IF > + 2swap fdt-prop > + THEN > + REPEAT > + r> > + drop > +; > + > +: fdt-flatten-node ( node -- ) > +\ fdt-debug IF dup node>path type cr THEN > + dup node>qname fdt-begin-node > + dup fdt-properties > + child > + BEGIN > + dup > + WHILE > + dup recurse > + peer > + REPEAT > + drop > + fdt-end-node > +; > + > +: fdt-strings-preload ( -- ) > + s" reg" fdt-add-string drop > + s" status" fdt-add-string drop > + s" 64-bit" fdt-add-string drop > + s" phandle" fdt-add-string drop > + s" ibm,vmx" fdt-add-string drop > + s" ibm,dfp" fdt-add-string drop > + s" slb-size" fdt-add-string drop > + s" ibm,purr" fdt-add-string drop > + s" vendor-id" fdt-add-string drop > + s" device-id" fdt-add-string drop > + s" min-grant" fdt-add-string drop > + s" class-code" fdt-add-string drop > + s" compatible" fdt-add-string drop > + s" interrupts" fdt-add-string drop > + s" cpu-version" fdt-add-string drop > + s" #size-cells" fdt-add-string drop > + s" ibm,req#msi" fdt-add-string drop > + s" revision-id" fdt-add-string drop > + s" device_type" fdt-add-string drop > + s" max-latency" fdt-add-string drop > + s" ibm,chip-id" fdt-add-string drop > + s" ibm,pft-size" fdt-add-string drop > + s" ibm,slb-size" fdt-add-string drop > + s" devsel-speed" fdt-add-string drop > + s" ibm,loc-code" fdt-add-string drop > + s" subsystem-id" fdt-add-string drop > + s" d-cache-size" fdt-add-string drop > + s" i-cache-size" fdt-add-string drop > + s" #address-cells" fdt-add-string drop > + s" clock-frequency" fdt-add-string drop > + s" cache-line-size" fdt-add-string drop > + s" ibm,pa-features" fdt-add-string drop > + s" ibm,my-drc-index" fdt-add-string drop > + s" d-cache-line-size" fdt-add-string drop > + s" i-cache-line-size" fdt-add-string drop > + s" assigned-addresses" fdt-add-string drop > + s" d-cache-block-size" fdt-add-string drop > + s" i-cache-block-size" fdt-add-string drop > + s" timebase-frequency" fdt-add-string drop > + s" subsystem-vendor-id" fdt-add-string drop > + s" ibm,segment-page-sizes" fdt-add-string drop > + s" ibm,ppc-interrupt-server#s" fdt-add-string drop > + s" ibm,processor-segment-sizes" fdt-add-string drop > + s" ibm,ppc-interrupt-gserver#s" fdt-add-string drop > +; > + > +: fdt-append-blob ( bytes cur blob -- cur ) > + 3dup -rot swap move > + drop + > +; > + > +: fdt-flatten-tree ( root -- tree ) > + 200000 alloc-mem dup fdt-struct-cur ! fdt-struct ! > + 200000 alloc-mem dup fdt-strings-cur ! fdt-strings ! > + > + fdt-debug IF > + 0 fdt-strings-reused ! > + milliseconds fdt-ms ! > + THEN > + > + \ Preload strings cache > + fdt-strings-preload > + fdt-strings-cur @ fdt-strings-cache ! > + \ Render the blobs > + fdt-flatten-node > + fdt-end > + > + \ Calculate strings and struct sizes > + fdt-struct-cur @ fdt-struct @ - > + fdt-strings-cur @ fdt-strings @ - ( struct-len strings-len ) > + > + 2dup + /fdth + > + 10 + \ Reserve 16 bytes and an empty reserved block > + > + fdt-debug IF > + 3dup > + ." FDT flat size=" .d cr > + ." Strings size=" .d cr > + ." Struct size=" .d cr > + ." Reused strings=" fdt-strings-reused @ .d cr > + milliseconds fdt-ms @ - > + ." Took " .d ." ms" cr > + THEN > + > + \ Allocate flatten DT blob > + dup alloc-mem ( struct-len strings-len total-len fdt ) > + >r ( struct-len strings-len total-len r: fdt ) > + > + \ Write header > + OF_DT_HEADER r@ >fdth_magic l! > + dup r@ >fdth_tsize l! > + /fdth 10 + 2 pick + r@ >fdth_struct_off l! > + /fdth 10 + r@ >fdth_string_off l! > + /fdth r@ >fdth_rsvmap_off l! > + 17 r@ >fdth_version l! > + 16 r@ >fdth_compat_vers l! > + 0 r@ >fdth_boot_cpu l! > + over r@ >fdth_string_size l! > + 2 pick r@ >fdth_struct_size l! > + ( struct-len strings-len total-len r: fdt ) > + > + drop ( struct-len strings-len r: fdt ) > + r@ /fdth + ( struct-len strings-len cur r: fdt ) > + > + \ Write the reserved entry > + 0 over ! > + cell+ > + 0 over ! > + cell+ ( struct-len strings-len cur r: fdt ) > + > + \ Write strings and struct blobs > + fdt-strings @ fdt-append-blob > + fdt-struct @ fdt-append-blob > + drop > + > + \ Free temporary blobs > + fdt-struct @ 200000 free-mem > + fdt-strings @ 200000 free-mem > + > + \ Return fdt > + r> > +; > + > +: fdt-flatten-tree-free ( tree ) > + dup >fdth_tsize l@ free-mem > +; > + > +: fdt ( -- ) > + " /" find-node > + fdt-flatten-tree > +; > + > s" /" find-node fdt-fix-phandles > diff --git a/board-qemu/slof/rtas.fs b/board-qemu/slof/rtas.fs > index 54d3929..5beb079 100644 > --- a/board-qemu/slof/rtas.fs > +++ b/board-qemu/slof/rtas.fs > @@ -98,6 +98,10 @@ find-qemu-rtas > ; > > : rtas-quiesce ( -- ) > + " /" find-node > + fdt-flatten-tree > + dup hv-update-dt > + fdt-flatten-tree-free > " quiesce" rtas-get-token rtas-cb rtas>token l! > 0 rtas-cb rtas>nargs l! > 0 rtas-cb rtas>nret l! > diff --git a/lib/libhvcall/hvcall.code b/lib/libhvcall/hvcall.code > index 8349748..6ff5715 100644 > --- a/lib/libhvcall/hvcall.code > +++ b/lib/libhvcall/hvcall.code > @@ -136,3 +136,8 @@ PRIM(hv_X2d_update_X2d_phandle) > uint32_t old_phandle = TOS.u; > TOS.u = hv_generic(KVMPPC_H_UPDATE_PHANDLE, old_phandle, new_phandle); > MIRP > + > +PRIM(hv_X2d_update_X2d_dt) > + unsigned long dt = TOS.u; > + TOS.u = hv_generic(KVMPPC_H_UPDATE_DT, dt); > +MIRP > diff --git a/lib/libhvcall/hvcall.in b/lib/libhvcall/hvcall.in > index ab7513a..b59e3f7 100644 > --- a/lib/libhvcall/hvcall.in > +++ b/lib/libhvcall/hvcall.in > @@ -32,4 +32,5 @@ cod(hv-logical-memop) > cod(hv-cas) > cod(hv-rtas-update) > cod(hv-update-phandle) > +cod(hv-update-dt) > cod(get-print-version)
On 03/10/17 09:39, Greg Kurz wrote: > On Tue, 3 Oct 2017 09:22:38 +1100 > Alexey Kardashevskiy <aik@ozlabs.ru> wrote: > >> On 03/10/17 03:18, Greg Kurz wrote: >>> On Mon, 2 Oct 2017 16:38:19 +1100 >>> Alexey Kardashevskiy <aik@ozlabs.ru> wrote: >>> >>>> This creates flatten device tree and passes it to QEMU via a custom >>>> hypercall right before jumping to RTAS. >>>> >>>> On a machine with 256 CPUs and 256 virtual Intel E1000 devices the blob >>>> is 360KB (356KB structs and 20KB of strings), building such a tree takes >>>> ~2s on a POWER8 box. A simple tree with 1 CPU and a couple of devices >>>> takes 38ms and creates 16KB blob. >>>> >>>> This preloads strings with 40 property names from CPU and PCI device nodes >>>> and the strings lookup only searches within these. Without string reusing >>>> at all, the strings blob is 200KB and rendering time is 1.7sec; with >>>> unlimited reusing, the strings blob is 4KB and rendering time is 2.8sec. >>>> >>>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru> >>>> --- >>>> >>>> Changes: >>>> v2: >>>> * fixed comments from review >>>> * added strings cache >>>> * changed last_compat_vers from 0x17 to 0x16 as suggested by dwg >>>> >>> >>> I wanted to give a try with PHB hotplug and SLOF breaks, because... >> >> How does it break exactly? I agree I do not clear the stack but it did not >> show up in any form :-/ >> > > Device tree strings 0x0000000004c90000 -> 0x0000000004c90a88 > Device tree struct 0x0000000004ca0000 -> 0x0000000004cb0000 > +++Q+++ (15990) h_update_dt 1676: DT at 7e43b000 (7e43b000) 19340 bytes > > > ( 700 ) Program Exception [ 7dc4b218 ] > > > R0 .. R7 R8 .. R15 R16 .. R23 R24 .. R31 > 000000007dbe2ea8 000000007e463010 0000000000000000 0000000000000006 > 000000007e666fe0 000000007dc4b220 0000000000000000 000000007dc05500 > 000000007dc0eb00 000000007e711048 000000007e463010 000000007dc090e8 > 000000007dc4d000 000000007dbe8510 000000007dc092b8 0000000000000003 > 000000007dc5f320 000000007dbe84f0 0000000000008000 000000000000f001 > 000000007dc4b218 0000000000000000 000000000000f003 ffffffffffffffff > 000000007e711050 0000000000000000 000000000000f004 000000007dc0ae08 > 0000000000000003 0000000000000000 000000000000f005 0000000000000000 > > CR / XER LR / CTR SRR0 / SRR1 DAR / DSISR > 80000408 000000007dbe3534 000000007dc0ae08 0000000000000000 > 0000000000000000 000000007dc0ae08 8000000000081000 00000000 > > > 45 > > > ( 300 ) Data Storage Exception [ 7dc4b240 ] > > > R0 .. R7 R8 .. R15 R16 .. R23 R24 .. R31 > 000000007dbe0308 000000007e463010 0000000000000000 0000000000000006 > 000000007e666fe0 4800000824000051 0000000000000000 000000007dc05500 > 000000007dc0eb00 000000007dc4b260 000000007e463010 000000007dc090e8 > 0000000000000064 000000007dc4eb00 000000007dc092b8 0000000000000003 > 0000000000000000 000000007dbe84f0 0000000000008000 000000000000f001 > 000000007dc4b240 0000000000000000 000000000000f003 ffffffffffffffff > 000000007dc09f30 0000000000000000 000000000000f004 000000007dbe3ba0 > 4800000824000051 0000000000000000 000000000000f005 000000007dc099a8 > > CR / XER LR / CTR SRR0 / SRR1 DAR / DSISR > 80000404 000000007dbe3490 000000007dbe42c4 4800000824000051 > 0000000020000000 000000007dbe3ba0 8000000000001000 40000000 > > I presume that this is the consequence of having 0 on top of the stack > when calling fdt-flatten-tree-free, or am I missing something ? It could be, I just wonder why it did not crash on my machine, it usually crashes when I screw up with the stack in forth :-/
On 03/10/17 09:48, Greg Kurz wrote: > On Mon, 2 Oct 2017 16:38:19 +1100 > Alexey Kardashevskiy <aik@ozlabs.ru> wrote: > >> This creates flatten device tree and passes it to QEMU via a custom >> hypercall right before jumping to RTAS. >> >> On a machine with 256 CPUs and 256 virtual Intel E1000 devices the blob >> is 360KB (356KB structs and 20KB of strings), building such a tree takes >> ~2s on a POWER8 box. A simple tree with 1 CPU and a couple of devices >> takes 38ms and creates 16KB blob. >> >> This preloads strings with 40 property names from CPU and PCI device nodes >> and the strings lookup only searches within these. Without string reusing >> at all, the strings blob is 200KB and rendering time is 1.7sec; with >> unlimited reusing, the strings blob is 4KB and rendering time is 2.8sec. >> >> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru> >> --- >> >> Changes: >> v2: >> * fixed comments from review >> * added strings cache >> * changed last_compat_vers from 0x17 to 0x16 as suggested by dwg >> > > It isn't 0x17 and 0x16 but 0x11 and 0x10 actually, fdt-flatten-tree > needs to be changed accordingly :) Hmmm. I looked at dtc - there the latest is 0d17 so you seem to be right. But when I did this patch, I just looked at the beginning of board-qemu/slof/fdt.fs: 53 : fdt-check-header ( -- ) [...] 72 dup >fdth_version l@ 17 >= IF 73 ." struct size : 0x" dup >fdth_struct_size l@ . cr 74 THEN Oopsie. I guess this should be 0x11 rather than 0x17, it is like that since the initial import.
On Tue, Oct 03, 2017 at 12:07:15PM +1100, Alexey Kardashevskiy wrote: > On 03/10/17 09:48, Greg Kurz wrote: > > On Mon, 2 Oct 2017 16:38:19 +1100 > > Alexey Kardashevskiy <aik@ozlabs.ru> wrote: > > > >> This creates flatten device tree and passes it to QEMU via a custom > >> hypercall right before jumping to RTAS. > >> > >> On a machine with 256 CPUs and 256 virtual Intel E1000 devices the blob > >> is 360KB (356KB structs and 20KB of strings), building such a tree takes > >> ~2s on a POWER8 box. A simple tree with 1 CPU and a couple of devices > >> takes 38ms and creates 16KB blob. > >> > >> This preloads strings with 40 property names from CPU and PCI device nodes > >> and the strings lookup only searches within these. Without string reusing > >> at all, the strings blob is 200KB and rendering time is 1.7sec; with > >> unlimited reusing, the strings blob is 4KB and rendering time is 2.8sec. > >> > >> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru> > >> --- > >> > >> Changes: > >> v2: > >> * fixed comments from review > >> * added strings cache > >> * changed last_compat_vers from 0x17 to 0x16 as suggested by dwg > >> > > > > It isn't 0x17 and 0x16 but 0x11 and 0x10 actually, fdt-flatten-tree > > needs to be changed accordingly :) > > Hmmm. I looked at dtc - there the latest is 0d17 so you seem to be right. > But when I did this patch, I just looked at the beginning of > board-qemu/slof/fdt.fs: > > > 53 : fdt-check-header ( -- ) > > [...] > 72 dup >fdth_version l@ 17 >= IF > > 73 ." struct size : 0x" dup >fdth_struct_size l@ . cr > > 74 THEN > > > > Oopsie. I guess this should be 0x11 rather than 0x17, it is like that since > the initial import. Uh, yes. I think you've gotten away with that because it just means SLOF would have ignored the extra v17 field, even on v17. That would work, the field is technically redundant, just means you can find the size of the structure block without having to scan right through it. Since SLOF will need to scan right through it anyway, it probably didn't really need it.
On Tue, 3 Oct 2017 12:07:15 +1100 Alexey Kardashevskiy <aik@ozlabs.ru> wrote: > On 03/10/17 09:48, Greg Kurz wrote: > > On Mon, 2 Oct 2017 16:38:19 +1100 > > Alexey Kardashevskiy <aik@ozlabs.ru> wrote: > > > >> This creates flatten device tree and passes it to QEMU via a custom > >> hypercall right before jumping to RTAS. > >> > >> On a machine with 256 CPUs and 256 virtual Intel E1000 devices the blob > >> is 360KB (356KB structs and 20KB of strings), building such a tree takes > >> ~2s on a POWER8 box. A simple tree with 1 CPU and a couple of devices > >> takes 38ms and creates 16KB blob. > >> > >> This preloads strings with 40 property names from CPU and PCI device nodes > >> and the strings lookup only searches within these. Without string reusing > >> at all, the strings blob is 200KB and rendering time is 1.7sec; with > >> unlimited reusing, the strings blob is 4KB and rendering time is 2.8sec. > >> > >> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru> > >> --- > >> > >> Changes: > >> v2: > >> * fixed comments from review > >> * added strings cache > >> * changed last_compat_vers from 0x17 to 0x16 as suggested by dwg > >> > > > > It isn't 0x17 and 0x16 but 0x11 and 0x10 actually, fdt-flatten-tree > > needs to be changed accordingly :) > > Hmmm. I looked at dtc - there the latest is 0d17 so you seem to be right. > But when I did this patch, I just looked at the beginning of > board-qemu/slof/fdt.fs: > > > 53 : fdt-check-header ( -- ) > > [...] > 72 dup >fdth_version l@ 17 >= IF > > 73 ." struct size : 0x" dup >fdth_struct_size l@ . cr > > 74 THEN > > > > Oopsie. I guess this should be 0x11 rather than 0x17, it is like that since > the initial import. I saw that too and yes, we're in hex mode so this should be 11. > > >
diff --git a/lib/libhvcall/libhvcall.h b/lib/libhvcall/libhvcall.h index 5776a2b..1c2d31b 100644 --- a/lib/libhvcall/libhvcall.h +++ b/lib/libhvcall/libhvcall.h @@ -26,6 +26,7 @@ #define KVMPPC_H_CAS (KVMPPC_HCALL_BASE + 0x2) #define KVMPPC_H_RTAS_UPDATE (KVMPPC_HCALL_BASE + 0x3) #define KVMPPC_H_UPDATE_PHANDLE (KVMPPC_HCALL_BASE + 0x4) +#define KVMPPC_H_UPDATE_DT (KVMPPC_HCALL_BASE + 0x5) #ifndef __ASSEMBLY__ diff --git a/board-qemu/slof/fdt.fs b/board-qemu/slof/fdt.fs index a24e344..ced9db3 100644 --- a/board-qemu/slof/fdt.fs +++ b/board-qemu/slof/fdt.fs @@ -27,7 +27,7 @@ struct 4 field >fdth_boot_cpu 4 field >fdth_string_size 4 field >fdth_struct_size -drop +constant /fdth h# d00dfeed constant OF_DT_HEADER h# 1 constant OF_DT_BEGIN_NODE @@ -449,4 +449,299 @@ r> drop fdt-cas-fix? ; +VARIABLE fdt-struct +VARIABLE fdt-struct-cur +VARIABLE fdt-strings +VARIABLE fdt-strings-cache +VARIABLE fdt-strings-cur +VARIABLE fdt-strings-reused +VARIABLE fdt-ms + +\ Adds data to struct blob, also adds terminating null if name is added +: fdt-struct-add ( bytes len nullterminate -- ) + >r + dup >r ( bytes len r: nullterminate len ) + fdt-struct-cur @ swap ( bytes cur len r: nullterminate len ) + \ copy bytes to structs + move + fdt-struct-cur @ ( cur r: nullterminate len ) + \ increment fdt-struct-cur + r> + ( cur r: nullterminate ) + \ add null if it is a string + r> IF + 0 over c! + 1+ + THEN + \ and align to 4 bytes + 3 + -4 and + fdt-struct-cur ! +; + +: fdt-encode-int ( token -- ) + fdt-struct-cur @ l! + /l fdt-struct-cur +! +; + +: fdt-skip-string ( cur -- cur ) + BEGIN + dup c@ + WHILE + 1+ + REPEAT + 4 + -4 and +; + +: zstring= ( str len zstr -- flag ) + 2dup + c@ 0<> IF + 3drop false + EXIT + THEN + swap comp 0= +; + +: fdt-find-string ( name namelen -- nameoff true | false ) + fdt-strings @ + BEGIN + dup fdt-strings-cache @ < + WHILE + 3dup zstring= IF + fdt-strings @ - + -rot + 2drop + true + EXIT + THEN + fdt-skip-string + REPEAT + 3drop + false +; + +: fdt-add-string ( name namelen -- nameoff ) + fdt-strings-cur @ + fdt-strings @ - >r ( name namelen r: nameoff ) + dup >r ( name namelen r: nameoff namelen ) + + \ copy string + fdt-strings-cur @ swap ( name cur namelen r: nameoff namelen ) + move ( r: nameoff namelen ) + + \ increment cur and null terminate + fdt-strings-cur @ + r> + ( cur r: nameoff ) + \ add null if it is a string + 0 over c! + \ add 1 byte for the null and align to 4 bytes + 4 + -4 and + fdt-strings-cur ! ( r: nameoff ) + r> ( nameoff ) +; + +: fdt-get-string ( name namelen -- nameoff ) + 2dup fdt-find-string IF + -rot 2drop + fdt-debug IF + 1 fdt-strings-reused +! + THEN + EXIT + THEN + fdt-add-string +; + +: fdt-begin-node ( name namelen -- ) + OF_DT_BEGIN_NODE fdt-encode-int + 2dup 1 = swap c@ [char] / = and \ is it "/"? + IF + 2drop s" " \ dtc is still unhappy though + THEN + true fdt-struct-add +; + +: fdt-end-node ( -- ) + OF_DT_END_NODE fdt-encode-int +; + +: fdt-prop ( prop len name namelen -- ) + OF_DT_PROP fdt-encode-int + + \ get string offset + fdt-get-string ( prop len nameoff ) + + \ store len and nameoff + over fdt-encode-int + fdt-encode-int ( prop len ) + + \ now add the bytes + false fdt-struct-add +; + +: fdt-end ( -- ) + OF_DT_END fdt-encode-int +; + +: fdt-properties ( phandle -- ) + dup encode-int s" phandle" fdt-prop + >r + s" " + BEGIN + r@ next-property + WHILE + 2dup + 2dup r@ get-property + not IF + 2swap fdt-prop + THEN + REPEAT + r> + drop +; + +: fdt-flatten-node ( node -- ) +\ fdt-debug IF dup node>path type cr THEN + dup node>qname fdt-begin-node + dup fdt-properties + child + BEGIN + dup + WHILE + dup recurse + peer + REPEAT + drop + fdt-end-node +; + +: fdt-strings-preload ( -- ) + s" reg" fdt-add-string drop + s" status" fdt-add-string drop + s" 64-bit" fdt-add-string drop + s" phandle" fdt-add-string drop + s" ibm,vmx" fdt-add-string drop + s" ibm,dfp" fdt-add-string drop + s" slb-size" fdt-add-string drop + s" ibm,purr" fdt-add-string drop + s" vendor-id" fdt-add-string drop + s" device-id" fdt-add-string drop + s" min-grant" fdt-add-string drop + s" class-code" fdt-add-string drop + s" compatible" fdt-add-string drop + s" interrupts" fdt-add-string drop + s" cpu-version" fdt-add-string drop + s" #size-cells" fdt-add-string drop + s" ibm,req#msi" fdt-add-string drop + s" revision-id" fdt-add-string drop + s" device_type" fdt-add-string drop + s" max-latency" fdt-add-string drop + s" ibm,chip-id" fdt-add-string drop + s" ibm,pft-size" fdt-add-string drop + s" ibm,slb-size" fdt-add-string drop + s" devsel-speed" fdt-add-string drop + s" ibm,loc-code" fdt-add-string drop + s" subsystem-id" fdt-add-string drop + s" d-cache-size" fdt-add-string drop + s" i-cache-size" fdt-add-string drop + s" #address-cells" fdt-add-string drop + s" clock-frequency" fdt-add-string drop + s" cache-line-size" fdt-add-string drop + s" ibm,pa-features" fdt-add-string drop + s" ibm,my-drc-index" fdt-add-string drop + s" d-cache-line-size" fdt-add-string drop + s" i-cache-line-size" fdt-add-string drop + s" assigned-addresses" fdt-add-string drop + s" d-cache-block-size" fdt-add-string drop + s" i-cache-block-size" fdt-add-string drop + s" timebase-frequency" fdt-add-string drop + s" subsystem-vendor-id" fdt-add-string drop + s" ibm,segment-page-sizes" fdt-add-string drop + s" ibm,ppc-interrupt-server#s" fdt-add-string drop + s" ibm,processor-segment-sizes" fdt-add-string drop + s" ibm,ppc-interrupt-gserver#s" fdt-add-string drop +; + +: fdt-append-blob ( bytes cur blob -- cur ) + 3dup -rot swap move + drop + +; + +: fdt-flatten-tree ( root -- tree ) + 200000 alloc-mem dup fdt-struct-cur ! fdt-struct ! + 200000 alloc-mem dup fdt-strings-cur ! fdt-strings ! + + fdt-debug IF + 0 fdt-strings-reused ! + milliseconds fdt-ms ! + THEN + + \ Preload strings cache + fdt-strings-preload + fdt-strings-cur @ fdt-strings-cache ! + \ Render the blobs + fdt-flatten-node + fdt-end + + \ Calculate strings and struct sizes + fdt-struct-cur @ fdt-struct @ - + fdt-strings-cur @ fdt-strings @ - ( struct-len strings-len ) + + 2dup + /fdth + + 10 + \ Reserve 16 bytes and an empty reserved block + + fdt-debug IF + 3dup + ." FDT flat size=" .d cr + ." Strings size=" .d cr + ." Struct size=" .d cr + ." Reused strings=" fdt-strings-reused @ .d cr + milliseconds fdt-ms @ - + ." Took " .d ." ms" cr + THEN + + \ Allocate flatten DT blob + dup alloc-mem ( struct-len strings-len total-len fdt ) + >r ( struct-len strings-len total-len r: fdt ) + + \ Write header + OF_DT_HEADER r@ >fdth_magic l! + dup r@ >fdth_tsize l! + /fdth 10 + 2 pick + r@ >fdth_struct_off l! + /fdth 10 + r@ >fdth_string_off l! + /fdth r@ >fdth_rsvmap_off l! + 17 r@ >fdth_version l! + 16 r@ >fdth_compat_vers l! + 0 r@ >fdth_boot_cpu l! + over r@ >fdth_string_size l! + 2 pick r@ >fdth_struct_size l! + ( struct-len strings-len total-len r: fdt ) + + drop ( struct-len strings-len r: fdt ) + r@ /fdth + ( struct-len strings-len cur r: fdt ) + + \ Write the reserved entry + 0 over ! + cell+ + 0 over ! + cell+ ( struct-len strings-len cur r: fdt ) + + \ Write strings and struct blobs + fdt-strings @ fdt-append-blob + fdt-struct @ fdt-append-blob + drop + + \ Free temporary blobs + fdt-struct @ 200000 free-mem + fdt-strings @ 200000 free-mem + + \ Return fdt + r> +; + +: fdt-flatten-tree-free ( tree ) + dup >fdth_tsize l@ free-mem +; + +: fdt ( -- ) + " /" find-node + fdt-flatten-tree +; + s" /" find-node fdt-fix-phandles diff --git a/board-qemu/slof/rtas.fs b/board-qemu/slof/rtas.fs index 54d3929..5beb079 100644 --- a/board-qemu/slof/rtas.fs +++ b/board-qemu/slof/rtas.fs @@ -98,6 +98,10 @@ find-qemu-rtas ; : rtas-quiesce ( -- ) + " /" find-node + fdt-flatten-tree + dup hv-update-dt + fdt-flatten-tree-free " quiesce" rtas-get-token rtas-cb rtas>token l! 0 rtas-cb rtas>nargs l! 0 rtas-cb rtas>nret l! diff --git a/lib/libhvcall/hvcall.code b/lib/libhvcall/hvcall.code index 8349748..6ff5715 100644 --- a/lib/libhvcall/hvcall.code +++ b/lib/libhvcall/hvcall.code @@ -136,3 +136,8 @@ PRIM(hv_X2d_update_X2d_phandle) uint32_t old_phandle = TOS.u; TOS.u = hv_generic(KVMPPC_H_UPDATE_PHANDLE, old_phandle, new_phandle); MIRP + +PRIM(hv_X2d_update_X2d_dt) + unsigned long dt = TOS.u; + TOS.u = hv_generic(KVMPPC_H_UPDATE_DT, dt); +MIRP diff --git a/lib/libhvcall/hvcall.in b/lib/libhvcall/hvcall.in index ab7513a..b59e3f7 100644 --- a/lib/libhvcall/hvcall.in +++ b/lib/libhvcall/hvcall.in @@ -32,4 +32,5 @@ cod(hv-logical-memop) cod(hv-cas) cod(hv-rtas-update) cod(hv-update-phandle) +cod(hv-update-dt) cod(get-print-version)
This creates flatten device tree and passes it to QEMU via a custom hypercall right before jumping to RTAS. On a machine with 256 CPUs and 256 virtual Intel E1000 devices the blob is 360KB (356KB structs and 20KB of strings), building such a tree takes ~2s on a POWER8 box. A simple tree with 1 CPU and a couple of devices takes 38ms and creates 16KB blob. This preloads strings with 40 property names from CPU and PCI device nodes and the strings lookup only searches within these. Without string reusing at all, the strings blob is 200KB and rendering time is 1.7sec; with unlimited reusing, the strings blob is 4KB and rendering time is 2.8sec. Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru> --- Changes: v2: * fixed comments from review * added strings cache * changed last_compat_vers from 0x17 to 0x16 as suggested by dwg --- I tested the blob by storing it from QEMU to a file and decompiling it; this produces error which I do not really understand as the name of the root is an empty string (literaly: 00 00 00 01 00 00 00 00) and yet this error: aik@fstn1-p1:~$ dtc -f -I dtb -O dts -o dbg.dts dbg.dtb ERROR (name_properties): "name" property in / is incorrect ("/" instead of base node name) Warning: Input tree has errors, output forced --- lib/libhvcall/libhvcall.h | 1 + board-qemu/slof/fdt.fs | 297 +++++++++++++++++++++++++++++++++++++++++++++- board-qemu/slof/rtas.fs | 4 + lib/libhvcall/hvcall.code | 5 + lib/libhvcall/hvcall.in | 1 + 5 files changed, 307 insertions(+), 1 deletion(-)