[bpf-next,0/6] Improve libbpf support of old kernels

This patch set improves libbpf's support of old kernels, missing features like
BTF support, global variables support, etc.

Most critical one is a silent drop of CO-RE relocations if libbpf fails to
load BTF (despite sanitization efforts). This is frequently the case for
kernels that have no BTF support whatsoever. There are still useful BPF
applications that could work on such kernels and do rely on CO-RE. To that
end, this series revamps the way BTF is handled in libbpf. Failure to load BTF
into kernel doesn't prevent libbpf from using BTF in its full capability
(e.g., for CO-RE relocations) internally.

Another issue that was identified was reliance of perf_buffer__new() on
BPF_OBJ_GET_INFO_BY_FD command, which is more recent that perf_buffer support
itself. Furthermore, BPF_OBJ_GET_INFO_BY_FD is needed just for some sanity
checks to provide better user errors, so could be safely omitted if kernel
doesn't provide it.

Perf_buffer selftest was adjusted to use skeleton, instead of bpf_prog_load().
The latter uses BPF_F_TEST_RND_HI32 flag, which is a relatively recent
addition and unnecessary fails selftest in libbpf's Travis CI tests. By using
skeleton we both get a shorter selftest and it work on pretty ancient kernels,
giving better libbpf test coverage.

One new selftest was added that relies on basic CO-RE features, but otherwise
doesn't expect any recent features (like global variables) from kernel. Again,
it's good to have better coverage of old kernels in libbpf testing.

Cc: Matthew Lim <matthewlim@fb.com>

Andrii Nakryiko (6):
  libbpf: make BTF finalization strict
  libbpf: add btf__set_fd() for more control over loaded BTF FD
  libbpf: improve BTF sanitization handling
  selftests/bpf: add test relying only on CO-RE and no recent kernel
    features
  libbpf: handle missing BPF_OBJ_GET_INFO_BY_FD gracefully in
    perf_buffer
  selftests/bpf: switch perf_buffer test to tracepoint and skeleton

 tools/lib/bpf/btf.c                           |   7 +-
 tools/lib/bpf/btf.h                           |   1 +
 tools/lib/bpf/libbpf.c                        | 150 ++++++++++--------
 tools/lib/bpf/libbpf.map                      |   1 +
 .../selftests/bpf/prog_tests/core_retro.c     |  33 ++++
 .../selftests/bpf/prog_tests/perf_buffer.c    |  42 ++---
 .../selftests/bpf/progs/test_core_retro.c     |  30 ++++
 .../selftests/bpf/progs/test_perf_buffer.c    |   4 +-
 8 files changed, 167 insertions(+), 101 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/core_retro.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_core_retro.c

On 7/8/20 3:53 AM, Andrii Nakryiko wrote:
> This patch set improves libbpf's support of old kernels, missing features like
> BTF support, global variables support, etc.
> 
> Most critical one is a silent drop of CO-RE relocations if libbpf fails to
> load BTF (despite sanitization efforts). This is frequently the case for
> kernels that have no BTF support whatsoever. There are still useful BPF
> applications that could work on such kernels and do rely on CO-RE. To that
> end, this series revamps the way BTF is handled in libbpf. Failure to load BTF
> into kernel doesn't prevent libbpf from using BTF in its full capability
> (e.g., for CO-RE relocations) internally.
> 
> Another issue that was identified was reliance of perf_buffer__new() on
> BPF_OBJ_GET_INFO_BY_FD command, which is more recent that perf_buffer support
> itself. Furthermore, BPF_OBJ_GET_INFO_BY_FD is needed just for some sanity
> checks to provide better user errors, so could be safely omitted if kernel
> doesn't provide it.
> 
> Perf_buffer selftest was adjusted to use skeleton, instead of bpf_prog_load().
> The latter uses BPF_F_TEST_RND_HI32 flag, which is a relatively recent
> addition and unnecessary fails selftest in libbpf's Travis CI tests. By using
> skeleton we both get a shorter selftest and it work on pretty ancient kernels,
> giving better libbpf test coverage.
> 
> One new selftest was added that relies on basic CO-RE features, but otherwise
> doesn't expect any recent features (like global variables) from kernel. Again,
> it's good to have better coverage of old kernels in libbpf testing.
> 
> Cc: Matthew Lim <matthewlim@fb.com>
> 
> Andrii Nakryiko (6):
>    libbpf: make BTF finalization strict
>    libbpf: add btf__set_fd() for more control over loaded BTF FD
>    libbpf: improve BTF sanitization handling
>    selftests/bpf: add test relying only on CO-RE and no recent kernel
>      features
>    libbpf: handle missing BPF_OBJ_GET_INFO_BY_FD gracefully in
>      perf_buffer
>    selftests/bpf: switch perf_buffer test to tracepoint and skeleton
> 
>   tools/lib/bpf/btf.c                           |   7 +-
>   tools/lib/bpf/btf.h                           |   1 +
>   tools/lib/bpf/libbpf.c                        | 150 ++++++++++--------
>   tools/lib/bpf/libbpf.map                      |   1 +
>   .../selftests/bpf/prog_tests/core_retro.c     |  33 ++++
>   .../selftests/bpf/prog_tests/perf_buffer.c    |  42 ++---
>   .../selftests/bpf/progs/test_core_retro.c     |  30 ++++
>   .../selftests/bpf/progs/test_perf_buffer.c    |   4 +-
>   8 files changed, 167 insertions(+), 101 deletions(-)
>   create mode 100644 tools/testing/selftests/bpf/prog_tests/core_retro.c
>   create mode 100644 tools/testing/selftests/bpf/progs/test_core_retro.c
> 

Applied, thanks!