| Message ID | 20190131065837.3380288-1-andriin@fb.com |
|---|---|
| Headers | show |
| Series | Add BTF types deduplication algorithm | expand |
add Edward. On 1/30/19 10:58 PM, Andrii Nakryiko wrote: > This patch series adds BTF deduplication algorithm to libbpf. This algorithm > allows to take BTF type information containing duplicate per-compilation unit > information and reduce it to equivalent set of BTF types with no duplication without > loss of information. It also deduplicates strings and removes those strings that > are not referenced from any BTF type (and line information in .BTF.ext section, > if any). > > Algorithm also resolves struct/union forward declarations into concrete BTF types > across multiple compilation units to facilitate better deduplication ratio. If > undesired, this resolution can be disabled through specifying corresponding options. > > When applied to BTF data emitted by pahole's DWARF->BTF converter, it reduces > the overall size of .BTF section by about 65x, from about 112MB to 1.75MB, leaving > only 29247 out of initial 3073497 BTF type descriptors. > > Algorithm with minor differences and preliminary results before FUNC/FUNC_PROTO > support is also described more verbosely at: > https://facebookmicrosites.github.io/bpf/blog/2018/11/14/btf-enhancement.html > > Andrii Nakryiko (3): > btf: extract BTF type size calculation > btf: add BTF types deduplication algorithm > selftests/btf: add initial BTF dedup tests > > tools/lib/bpf/btf.c | 1851 +++++++++++++++++++++++- > tools/lib/bpf/btf.h | 11 + > tools/lib/bpf/libbpf.map | 3 + > tools/testing/selftests/bpf/test_btf.c | 535 ++++++- > 4 files changed, 2333 insertions(+), 67 deletions(-) >
Hi Andrii, On 01/31/2019 07:58 AM, Andrii Nakryiko wrote: > This patch series adds BTF deduplication algorithm to libbpf. This algorithm > allows to take BTF type information containing duplicate per-compilation unit > information and reduce it to equivalent set of BTF types with no duplication without > loss of information. It also deduplicates strings and removes those strings that > are not referenced from any BTF type (and line information in .BTF.ext section, > if any). > > Algorithm also resolves struct/union forward declarations into concrete BTF types > across multiple compilation units to facilitate better deduplication ratio. If > undesired, this resolution can be disabled through specifying corresponding options. > > When applied to BTF data emitted by pahole's DWARF->BTF converter, it reduces > the overall size of .BTF section by about 65x, from about 112MB to 1.75MB, leaving > only 29247 out of initial 3073497 BTF type descriptors. > > Algorithm with minor differences and preliminary results before FUNC/FUNC_PROTO > support is also described more verbosely at: > https://facebookmicrosites.github.io/bpf/blog/2018/11/14/btf-enhancement.html > > Andrii Nakryiko (3): > btf: extract BTF type size calculation > btf: add BTF types deduplication algorithm > selftests/btf: add initial BTF dedup tests > > tools/lib/bpf/btf.c | 1851 +++++++++++++++++++++++- > tools/lib/bpf/btf.h | 11 + > tools/lib/bpf/libbpf.map | 3 + > tools/testing/selftests/bpf/test_btf.c | 535 ++++++- > 4 files changed, 2333 insertions(+), 67 deletions(-) This would need a proper rebase for bpf-next so that it applies w/o bigger conflicts. Please also change the libbpf.map and place the newly exported functions under LIBBPF_0.0.2 (as everything under 0.0.1 was from prior released kernel). Thanks, Daniel
On Mon, Feb 4, 2019 at 5:32 PM Daniel Borkmann <daniel@iogearbox.net> wrote: > > Hi Andrii, > > On 01/31/2019 07:58 AM, Andrii Nakryiko wrote: > > This patch series adds BTF deduplication algorithm to libbpf. This algorithm > > allows to take BTF type information containing duplicate per-compilation unit > > information and reduce it to equivalent set of BTF types with no duplication without > > loss of information. It also deduplicates strings and removes those strings that > > are not referenced from any BTF type (and line information in .BTF.ext section, > > if any). > > > > Algorithm also resolves struct/union forward declarations into concrete BTF types > > across multiple compilation units to facilitate better deduplication ratio. If > > undesired, this resolution can be disabled through specifying corresponding options. > > > > When applied to BTF data emitted by pahole's DWARF->BTF converter, it reduces > > the overall size of .BTF section by about 65x, from about 112MB to 1.75MB, leaving > > only 29247 out of initial 3073497 BTF type descriptors. > > > > Algorithm with minor differences and preliminary results before FUNC/FUNC_PROTO > > support is also described more verbosely at: > > https://facebookmicrosites.github.io/bpf/blog/2018/11/14/btf-enhancement.html > > > > Andrii Nakryiko (3): > > btf: extract BTF type size calculation > > btf: add BTF types deduplication algorithm > > selftests/btf: add initial BTF dedup tests > > > > tools/lib/bpf/btf.c | 1851 +++++++++++++++++++++++- > > tools/lib/bpf/btf.h | 11 + > > tools/lib/bpf/libbpf.map | 3 + > > tools/testing/selftests/bpf/test_btf.c | 535 ++++++- > > 4 files changed, 2333 insertions(+), 67 deletions(-) > > This would need a proper rebase for bpf-next so that it applies w/o > bigger conflicts. Please also change the libbpf.map and place the newly > exported functions under LIBBPF_0.0.2 (as everything under 0.0.1 was from > prior released kernel). Done. Thanks! > > Thanks, > Daniel