| Message ID | 20200619230423.691274-1-andriin@fb.com |
|---|---|
| State | Accepted |
| Delegated to: | BPF Maintainers |
| Headers | show |
| Series | [bpf] libbpf: fix CO-RE relocs against .text section | expand |
On 6/19/20 4:04 PM, Andrii Nakryiko wrote: > bpf_object__find_program_by_title(), used by CO-RE relocation code, doesn't > return .text "BPF program", if it is a function storage for sub-programs. > Because of that, any CO-RE relocation in helper non-inlined functions will > fail. Fix this by searching for .text-corresponding BPF program manually. > > Adjust one of bpf_iter selftest to exhibit this pattern. > > Reported-by: Yonghong Song <yhs@fb.com> > Fixes: ddc7c3042614 ("libbpf: implement BPF CO-RE offset relocation algorithm") > Signed-off-by: Andrii Nakryiko <andriin@fb.com> Acked-by: Yonghong Song <yhs@fb.com> But the fix here only fixed the issue for interpreter mode. For jit only mode, we still have issues. The following patch can fix the jit mode issue, ============= From 4d66814513ec45b86a30a1231b8a000d4bfc6f1a Mon Sep 17 00:00:00 2001 From: Yonghong Song <yhs@fb.com> Date: Fri, 19 Jun 2020 23:26:13 -0700 Subject: [PATCH bpf] bpf: set the number of exception entries properly for subprograms Currently, if a bpf program has more than one subprograms, each program will be jitted separately. For tracing problem, the prog->aux->num_exentries is not setup properly. For example, with bpf_iter_netlink.c modified to force one function not inlined, and with proper libbpf fix, with CONFIG_BPF_JIT_ALWAYS_ON, we will have error like below: $ ./test_progs -n 3/3 ... libbpf: failed to load program 'iter/netlink' libbpf: failed to load object 'bpf_iter_netlink' libbpf: failed to load BPF skeleton 'bpf_iter_netlink': -4007 test_netlink:FAIL:bpf_iter_netlink__open_and_load skeleton open_and_load failed #3/3 netlink:FAIL The dmesg shows the following errors: ex gen bug which is triggered by the following code in arch/x86/net/bpf_jit_comp.c: if (excnt >= bpf_prog->aux->num_exentries) { pr_err("ex gen bug\n"); return -EFAULT; } If the program has more than one subprograms, num_exentries is actually 0 since it is not setup. This patch fixed the issue by setuping proper num_exentries for each subprogram before calling jit function. Signed-off-by: Yonghong Song <yhs@fb.com> --- kernel/bpf/verifier.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 34cde841ab68..7d8b23ba825c 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -9801,7 +9801,7 @@ static int jit_subprogs(struct bpf_verifier_env *env) int i, j, subprog_start, subprog_end = 0, len, subprog; struct bpf_insn *insn; void *old_bpf_func; - int err; + int err, num_exentries; if (env->subprog_cnt <= 1) return 0; @@ -9876,6 +9876,16 @@ static int jit_subprogs(struct bpf_verifier_env *env) func[i]->aux->nr_linfo = prog->aux->nr_linfo; func[i]->aux->jited_linfo = prog->aux->jited_linfo; func[i]->aux->linfo_idx = env->subprog_info[i].linfo_idx; + + num_exentries = 0; + insn = func[i]->insnsi; + for (j = 0; j < func[i]->len; j++, insn++) { + if (BPF_CLASS(insn->code) == BPF_LDX && + BPF_MODE(insn->code) == BPF_PROBE_MEM) + num_exentries++; + } + func[i]->aux->num_exentries = num_exentries; + func[i] = bpf_int_jit_compile(func[i]); if (!func[i]->jited) { err = -ENOTSUPP;
On Sat, Jun 20, 2020 at 12:06 AM Yonghong Song <yhs@fb.com> wrote: > > > > On 6/19/20 4:04 PM, Andrii Nakryiko wrote: > > bpf_object__find_program_by_title(), used by CO-RE relocation code, doesn't > > return .text "BPF program", if it is a function storage for sub-programs. > > Because of that, any CO-RE relocation in helper non-inlined functions will > > fail. Fix this by searching for .text-corresponding BPF program manually. > > > > Adjust one of bpf_iter selftest to exhibit this pattern. > > > > Reported-by: Yonghong Song <yhs@fb.com> > > Fixes: ddc7c3042614 ("libbpf: implement BPF CO-RE offset relocation algorithm") > > Signed-off-by: Andrii Nakryiko <andriin@fb.com> > > Acked-by: Yonghong Song <yhs@fb.com> > > But the fix here only fixed the issue for interpreter mode. > For jit only mode, we still have issues. The following patch can fix > the jit mode issue, > > ============= > > From 4d66814513ec45b86a30a1231b8a000d4bfc6f1a Mon Sep 17 00:00:00 2001 > From: Yonghong Song <yhs@fb.com> > Date: Fri, 19 Jun 2020 23:26:13 -0700 > Subject: [PATCH bpf] bpf: set the number of exception entries properly for > subprograms > > Currently, if a bpf program has more than one subprograms, each > program will be jitted separately. For tracing problem, the > prog->aux->num_exentries is not setup properly. For example, > with bpf_iter_netlink.c modified to force one function not inlined, > and with proper libbpf fix, with CONFIG_BPF_JIT_ALWAYS_ON, > we will have error like below: > $ ./test_progs -n 3/3 > ... > libbpf: failed to load program 'iter/netlink' > libbpf: failed to load object 'bpf_iter_netlink' > libbpf: failed to load BPF skeleton 'bpf_iter_netlink': -4007 > test_netlink:FAIL:bpf_iter_netlink__open_and_load skeleton > open_and_load failed > #3/3 netlink:FAIL > The dmesg shows the following errors: > ex gen bug > which is triggered by the following code in arch/x86/net/bpf_jit_comp.c: > if (excnt >= bpf_prog->aux->num_exentries) { > pr_err("ex gen bug\n"); > return -EFAULT; > } > > If the program has more than one subprograms, num_exentries is actually > 0 since it is not setup. > > This patch fixed the issue by setuping proper num_exentries for > each subprogram before calling jit function. > > Signed-off-by: Yonghong Song <yhs@fb.com> Thanks for fixing. Applied both to bpf tree. Yonghong, next time please submit the patch properly. It was very awkward to copy-paste it manually from the thread. I've edited the commit log a bit.
On 6/23/20 5:40 PM, Alexei Starovoitov wrote: > On Sat, Jun 20, 2020 at 12:06 AM Yonghong Song <yhs@fb.com> wrote: >> >> >> >> On 6/19/20 4:04 PM, Andrii Nakryiko wrote: >>> bpf_object__find_program_by_title(), used by CO-RE relocation code, doesn't >>> return .text "BPF program", if it is a function storage for sub-programs. >>> Because of that, any CO-RE relocation in helper non-inlined functions will >>> fail. Fix this by searching for .text-corresponding BPF program manually. >>> >>> Adjust one of bpf_iter selftest to exhibit this pattern. >>> >>> Reported-by: Yonghong Song <yhs@fb.com> >>> Fixes: ddc7c3042614 ("libbpf: implement BPF CO-RE offset relocation algorithm") >>> Signed-off-by: Andrii Nakryiko <andriin@fb.com> >> >> Acked-by: Yonghong Song <yhs@fb.com> >> >> But the fix here only fixed the issue for interpreter mode. >> For jit only mode, we still have issues. The following patch can fix >> the jit mode issue, >> >> ============= >> >> From 4d66814513ec45b86a30a1231b8a000d4bfc6f1a Mon Sep 17 00:00:00 2001 >> From: Yonghong Song <yhs@fb.com> >> Date: Fri, 19 Jun 2020 23:26:13 -0700 >> Subject: [PATCH bpf] bpf: set the number of exception entries properly for >> subprograms >> >> Currently, if a bpf program has more than one subprograms, each >> program will be jitted separately. For tracing problem, the >> prog->aux->num_exentries is not setup properly. For example, >> with bpf_iter_netlink.c modified to force one function not inlined, >> and with proper libbpf fix, with CONFIG_BPF_JIT_ALWAYS_ON, >> we will have error like below: >> $ ./test_progs -n 3/3 >> ... >> libbpf: failed to load program 'iter/netlink' >> libbpf: failed to load object 'bpf_iter_netlink' >> libbpf: failed to load BPF skeleton 'bpf_iter_netlink': -4007 >> test_netlink:FAIL:bpf_iter_netlink__open_and_load skeleton >> open_and_load failed >> #3/3 netlink:FAIL >> The dmesg shows the following errors: >> ex gen bug >> which is triggered by the following code in arch/x86/net/bpf_jit_comp.c: >> if (excnt >= bpf_prog->aux->num_exentries) { >> pr_err("ex gen bug\n"); >> return -EFAULT; >> } >> >> If the program has more than one subprograms, num_exentries is actually >> 0 since it is not setup. >> >> This patch fixed the issue by setuping proper num_exentries for >> each subprogram before calling jit function. >> >> Signed-off-by: Yonghong Song <yhs@fb.com> > > Thanks for fixing. Applied both to bpf tree. > Yonghong, next time please submit the patch properly. > It was very awkward to copy-paste it manually from the thread. > I've edited the commit log a bit. Thanks. I posted original commit as I am not sure how to proceed as this and Andrii's patch belongs to the same patch set to fix bpf_iter_netlink problem. I guess next time I will go ahead with patch submit with proper description in the patch, which sounds better for review and to get notice from other people.
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 477c679ed945..f17151d866e6 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -4818,7 +4818,13 @@ bpf_core_reloc_fields(struct bpf_object *obj, const char *targ_btf_path) err = -EINVAL; goto out; } - prog = bpf_object__find_program_by_title(obj, sec_name); + prog = NULL; + for (i = 0; i < obj->nr_programs; i++) { + if (!strcmp(obj->programs[i].section_name, sec_name)) { + prog = &obj->programs[i]; + break; + } + } if (!prog) { pr_warn("failed to find program '%s' for CO-RE offset relocation\n", sec_name); diff --git a/tools/testing/selftests/bpf/progs/bpf_iter_netlink.c b/tools/testing/selftests/bpf/progs/bpf_iter_netlink.c index e7b8753eac0b..75ecf956a2df 100644 --- a/tools/testing/selftests/bpf/progs/bpf_iter_netlink.c +++ b/tools/testing/selftests/bpf/progs/bpf_iter_netlink.c @@ -25,7 +25,7 @@ struct bpf_iter__netlink { struct netlink_sock *sk; } __attribute__((preserve_access_index)); -static inline struct inode *SOCK_INODE(struct socket *socket) +static __attribute__((noinline)) struct inode *SOCK_INODE(struct socket *socket) { return &container_of(socket, struct socket_alloc, socket)->vfs_inode; }
bpf_object__find_program_by_title(), used by CO-RE relocation code, doesn't return .text "BPF program", if it is a function storage for sub-programs. Because of that, any CO-RE relocation in helper non-inlined functions will fail. Fix this by searching for .text-corresponding BPF program manually. Adjust one of bpf_iter selftest to exhibit this pattern. Reported-by: Yonghong Song <yhs@fb.com> Fixes: ddc7c3042614 ("libbpf: implement BPF CO-RE offset relocation algorithm") Signed-off-by: Andrii Nakryiko <andriin@fb.com> --- tools/lib/bpf/libbpf.c | 8 +++++++- tools/testing/selftests/bpf/progs/bpf_iter_netlink.c | 2 +- 2 files changed, 8 insertions(+), 2 deletions(-)