| Message ID | 20200813071722.2213397-1-andriin@fb.com |
|---|---|
| Headers | show |
| Series | Fix various issues with 32-bit libbpf | expand |
On Thu, Aug 13, 2020 at 12:17:13AM -0700, Andrii Nakryiko wrote: > This patch set contains fixes to libbpf, bpftool, and selftests that were > found while testing libbpf and selftests built in 32-bit mode. 64-bit nature > of BPF target and 32-bit host environment don't always mix together well > without extra care, so there were a bunch of problems discovered and fixed. > > Each individual patch contains additional explanations, where necessary. > > This series is really a mix of bpf tree fixes and patches that are better > landed into bpf-next, once it opens. This is due to a bit riskier changes and > new APIs added to allow solving this 32/64-bit mix problem. It would be great > to apply patches #1 through #3 to bpf tree right now, and the rest into > bpf-next, but I would appreciate reviewing all of them, of course. why first three only? I think btf__set_pointer_size() and friends are necessary in bpf tree. The only thing I would suggest is to rename guess_ptr_size() into determine_ptr_size() or something. It's not guessing it. Looking for 'long' in BTF is precise. We can teach pahole and llvm to always emit 'long' type and libbpf can fail parsing BTF if 'long' is not found.
On Thu, Aug 13, 2020 at 12:20 PM Alexei Starovoitov <alexei.starovoitov@gmail.com> wrote: > > On Thu, Aug 13, 2020 at 12:17:13AM -0700, Andrii Nakryiko wrote: > > This patch set contains fixes to libbpf, bpftool, and selftests that were > > found while testing libbpf and selftests built in 32-bit mode. 64-bit nature > > of BPF target and 32-bit host environment don't always mix together well > > without extra care, so there were a bunch of problems discovered and fixed. > > > > Each individual patch contains additional explanations, where necessary. > > > > This series is really a mix of bpf tree fixes and patches that are better > > landed into bpf-next, once it opens. This is due to a bit riskier changes and > > new APIs added to allow solving this 32/64-bit mix problem. It would be great > > to apply patches #1 through #3 to bpf tree right now, and the rest into > > bpf-next, but I would appreciate reviewing all of them, of course. > > why first three only? > I think btf__set_pointer_size() and friends are necessary in bpf tree. I don't mind. The "scariest" change is bpftool's skeleton generation change, so would be good if you double-check the logic of enforcing at most 4 byte alignment. But it seems logically sound and safe to me. > The only thing I would suggest is to rename guess_ptr_size() into > determine_ptr_size() or something. > It's not guessing it. Looking for 'long' in BTF is precise. It was a guess only in the sense that it won't work for LLP64 model, for instance. But that model is used on Windows platforms, it seems. Linux sticks to ILP32 and LP64, both of which have sizeof(long) == sizeof(void *). I'll rename it to determine_ptr_sz() then. > We can teach pahole and llvm to always emit 'long' type and libbpf can > fail parsing BTF if 'long' is not found. It's always the case for vmlinux BTF, so no need there. As for BPF .o files, libbpf just enforces 64-bit pointer size anyways. So all typical cases are covered reliably, I think. It's only for some non-mainstream use cases where people want to use libbpf's BTF API for some unconventional uses of BTF. I don't know any of such cases besides our own selftests, but it's a generic library API, so who knows :) We also can do detection based on ELF target architecture, for btf__parse_elf() API. btf__parse_raw() is a bit less lucky in that regard.
This patch set contains fixes to libbpf, bpftool, and selftests that were found while testing libbpf and selftests built in 32-bit mode. 64-bit nature of BPF target and 32-bit host environment don't always mix together well without extra care, so there were a bunch of problems discovered and fixed. Each individual patch contains additional explanations, where necessary. This series is really a mix of bpf tree fixes and patches that are better landed into bpf-next, once it opens. This is due to a bit riskier changes and new APIs added to allow solving this 32/64-bit mix problem. It would be great to apply patches #1 through #3 to bpf tree right now, and the rest into bpf-next, but I would appreciate reviewing all of them, of course. Andrii Nakryiko (9): tools/bpftool: fix compilation warnings in 32-bit mode selftest/bpf: fix compilation warnings in 32-bit mode libbpf: fix BTF-defined map-in-map initialization on 32-bit host arches libbpf: handle BTF pointer sizes more carefully selftests/bpf: fix btf_dump test cases on 32-bit arches libbpf: enforce 64-bitness of BTF for BPF object files selftests/bpf: correct various core_reloc 64-bit assumptions tools/bpftool: generate data section struct with conservative alignment selftests/bpf: make test_varlen work with 32-bit user-space arch tools/bpf/bpftool/btf_dumper.c | 2 +- tools/bpf/bpftool/gen.c | 14 ++++ tools/bpf/bpftool/link.c | 4 +- tools/bpf/bpftool/main.h | 10 ++- tools/bpf/bpftool/prog.c | 16 ++--- tools/lib/bpf/btf.c | 71 ++++++++++++++++++- tools/lib/bpf/btf.h | 2 + tools/lib/bpf/btf_dump.c | 4 +- tools/lib/bpf/libbpf.c | 20 ++++-- tools/lib/bpf/libbpf.map | 2 + .../selftests/bpf/prog_tests/bpf_obj_id.c | 8 +-- .../selftests/bpf/prog_tests/btf_dump.c | 27 +++++-- .../selftests/bpf/prog_tests/core_extern.c | 4 +- .../selftests/bpf/prog_tests/core_reloc.c | 20 +++--- .../selftests/bpf/prog_tests/fexit_bpf2bpf.c | 6 +- .../selftests/bpf/prog_tests/flow_dissector.c | 2 +- .../selftests/bpf/prog_tests/global_data.c | 6 +- .../selftests/bpf/prog_tests/prog_run_xattr.c | 2 +- .../selftests/bpf/prog_tests/skb_ctx.c | 2 +- .../testing/selftests/bpf/prog_tests/varlen.c | 8 +-- .../selftests/bpf/progs/core_reloc_types.h | 69 +++++++++--------- .../testing/selftests/bpf/progs/test_varlen.c | 6 +- tools/testing/selftests/bpf/test_btf.c | 8 +-- tools/testing/selftests/bpf/test_progs.h | 5 ++ 24 files changed, 221 insertions(+), 97 deletions(-)