| Message ID | 20200715233634.3868-4-maciej.fijalkowski@intel.com |
|---|---|
| State | Changes Requested |
| Delegated to: | BPF Maintainers |
| Headers | show |
| Series | bpf: tailcalls in BPF subprograms | expand |
On 7/16/20 1:36 AM, Maciej Fijalkowski wrote: > Previously, there was no need for poke descriptors being present in > subprogram's bpf_prog_aux struct since tailcalls were simply not allowed > in them. Each subprog is JITed independently so in order to enable > JITing such subprograms, simply copy poke descriptors from main program > to subprogram's poke tab. > > Add also subprog's aux struct to the BPF map poke_progs list by calling > on it map_poke_track(). > > Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com> > --- > kernel/bpf/verifier.c | 9 +++++++++ > 1 file changed, 9 insertions(+) > > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c > index 6481342b31ba..3b406b2860ef 100644 > --- a/kernel/bpf/verifier.c > +++ b/kernel/bpf/verifier.c > @@ -9932,6 +9932,9 @@ static int jit_subprogs(struct bpf_verifier_env *env) > goto out_undo_insn; > > for (i = 0; i < env->subprog_cnt; i++) { > + struct bpf_map *map_ptr; > + int j; > + > subprog_start = subprog_end; > subprog_end = env->subprog_info[i + 1].start; > > @@ -9956,6 +9959,12 @@ static int jit_subprogs(struct bpf_verifier_env *env) > func[i]->aux->btf = prog->aux->btf; > func[i]->aux->func_info = prog->aux->func_info; > > + for (j = 0; j < prog->aux->size_poke_tab; j++) { > + bpf_jit_add_poke_descriptor(func[i], &prog->aux->poke_tab[j]); > + map_ptr = func[i]->aux->poke_tab[j].tail_call.map; > + map_ptr->ops->map_poke_track(map_ptr, func[i]->aux); Error checking missing for bpf_jit_add_poke_descriptor() and map_poke_track() ..? It must be guaranteed that adding this to the tracker must not fail, otherwise this will be a real pain to debug given the prog will never be patched. > + } > + > /* Use bpf_prog_F_tag to indicate functions in stack traces. > * Long term would need debug info to populate names > */ >
On Thu, Jul 16, 2020 at 11:18 PM Daniel Borkmann <daniel@iogearbox.net> wrote: > > On 7/16/20 1:36 AM, Maciej Fijalkowski wrote: > > Previously, there was no need for poke descriptors being present in > > subprogram's bpf_prog_aux struct since tailcalls were simply not allowed > > in them. Each subprog is JITed independently so in order to enable > > JITing such subprograms, simply copy poke descriptors from main program > > to subprogram's poke tab. > > > > Add also subprog's aux struct to the BPF map poke_progs list by calling > > on it map_poke_track(). > > > > Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com> > > --- > > kernel/bpf/verifier.c | 9 +++++++++ > > 1 file changed, 9 insertions(+) > > > > diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c > > index 6481342b31ba..3b406b2860ef 100644 > > --- a/kernel/bpf/verifier.c > > +++ b/kernel/bpf/verifier.c > > @@ -9932,6 +9932,9 @@ static int jit_subprogs(struct bpf_verifier_env *env) > > goto out_undo_insn; > > > > for (i = 0; i < env->subprog_cnt; i++) { > > + struct bpf_map *map_ptr; > > + int j; > > + > > subprog_start = subprog_end; > > subprog_end = env->subprog_info[i + 1].start; > > > > @@ -9956,6 +9959,12 @@ static int jit_subprogs(struct bpf_verifier_env *env) > > func[i]->aux->btf = prog->aux->btf; > > func[i]->aux->func_info = prog->aux->func_info; > > > > + for (j = 0; j < prog->aux->size_poke_tab; j++) { > > + bpf_jit_add_poke_descriptor(func[i], &prog->aux->poke_tab[j]); > > + map_ptr = func[i]->aux->poke_tab[j].tail_call.map; > > + map_ptr->ops->map_poke_track(map_ptr, func[i]->aux); > > Error checking missing for bpf_jit_add_poke_descriptor() and map_poke_track() ..? It > must be guaranteed that adding this to the tracker must not fail, otherwise this will > be a real pain to debug given the prog will never be patched. My bad, will fix it in v2. > > > + } > > + > > /* Use bpf_prog_F_tag to indicate functions in stack traces. > > * Long term would need debug info to populate names > > */ > > >
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 6481342b31ba..3b406b2860ef 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -9932,6 +9932,9 @@ static int jit_subprogs(struct bpf_verifier_env *env) goto out_undo_insn; for (i = 0; i < env->subprog_cnt; i++) { + struct bpf_map *map_ptr; + int j; + subprog_start = subprog_end; subprog_end = env->subprog_info[i + 1].start; @@ -9956,6 +9959,12 @@ static int jit_subprogs(struct bpf_verifier_env *env) func[i]->aux->btf = prog->aux->btf; func[i]->aux->func_info = prog->aux->func_info; + for (j = 0; j < prog->aux->size_poke_tab; j++) { + bpf_jit_add_poke_descriptor(func[i], &prog->aux->poke_tab[j]); + map_ptr = func[i]->aux->poke_tab[j].tail_call.map; + map_ptr->ops->map_poke_track(map_ptr, func[i]->aux); + } + /* Use bpf_prog_F_tag to indicate functions in stack traces. * Long term would need debug info to populate names */
Previously, there was no need for poke descriptors being present in subprogram's bpf_prog_aux struct since tailcalls were simply not allowed in them. Each subprog is JITed independently so in order to enable JITing such subprograms, simply copy poke descriptors from main program to subprogram's poke tab. Add also subprog's aux struct to the BPF map poke_progs list by calling on it map_poke_track(). Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com> --- kernel/bpf/verifier.c | 9 +++++++++ 1 file changed, 9 insertions(+)