| Message ID | 20200521191752.3448223-1-kafai@fb.com |
|---|---|
| Headers | show |
| Series | bpf: Allow inner map with different max_entries | expand |
On Thu, May 21, 2020 at 12:18 PM Martin KaFai Lau <kafai@fb.com> wrote: > > This series allows the outer map to be updated with inner map in different > size as long as it is safe (meaning the max_entries is not used in the > verification time during prog load). > > Please see individual patch for details. > Few thoughts: 1. You describe WHAT, but not necessarily WHY. Can you please elaborate in descriptions what motivates these changes? 2. IMO, "capabilities" is word that way too strongly correlates with Linux capabilities framework, it's just confusing. It's also more of a property of a map type, than what map is capable of, but it's more philosophical distinction, of course :) 3. I'm honestly not convinced that patch #1 qualifies as a clean up. I think one specific check for types of maps that are not compatible with map-in-map is just fine. Instead you are spreading this bit flags into a long list of maps, most of which ARE compatible. It's just hard to even see which ones are not compatible. I like current way better. 4. Then for size check change, again, it's really much simpler and cleaner just to have a special case in check in bpf_map_meta_equal for cases where map size matters. 5. I also wonder if for those inner maps for which size doesn't matter, maybe we should set max_elements to zero when setting inner_meta to show that size doesn't matter? This is minor, though. > Martin KaFai Lau (3): > bpf: Clean up inner map type check > bpf: Relax the max_entries check for inner map > bpf: selftests: Add test for different inner map size > > include/linux/bpf.h | 18 +++++- > include/linux/bpf_types.h | 64 +++++++++++-------- > kernel/bpf/btf.c | 2 +- > kernel/bpf/map_in_map.c | 12 ++-- > kernel/bpf/syscall.c | 19 +++++- > kernel/bpf/verifier.c | 2 +- > .../selftests/bpf/prog_tests/btf_map_in_map.c | 12 ++++ > .../selftests/bpf/progs/test_btf_map_in_map.c | 31 +++++++++ > 8 files changed, 119 insertions(+), 41 deletions(-) > > -- > 2.24.1 >
On Thu, May 21, 2020 at 03:39:10PM -0700, Andrii Nakryiko wrote: > On Thu, May 21, 2020 at 12:18 PM Martin KaFai Lau <kafai@fb.com> wrote: > > > > This series allows the outer map to be updated with inner map in different > > size as long as it is safe (meaning the max_entries is not used in the > > verification time during prog load). > > > > Please see individual patch for details. > > > > Few thoughts: > > 1. You describe WHAT, but not necessarily WHY. Can you please > elaborate in descriptions what motivates these changes? There are cases where people want to update a bigger size inner map. I will update the cover letter. > 2. IMO, "capabilities" is word that way too strongly correlates with > Linux capabilities framework, it's just confusing. It's also more of a > property of a map type, than what map is capable of, but it's more > philosophical distinction, of course :) Sure. I can rename it to "property" > 3. I'm honestly not convinced that patch #1 qualifies as a clean up. I > think one specific check for types of maps that are not compatible > with map-in-map is just fine. Instead you are spreading this bit flags > into a long list of maps, most of which ARE compatible. but in one place and at the same time a new map type is added to bpf_types.h > It's just hard > to even see which ones are not compatible. I like current way better. There are multiple cases that people forgot to exclude a new map type from map-in-map in the first attempt and fix it up later. During the map-in-map implementation, this same concern was raised also about how to better exclude future map type from map-in-map since not all people has used map-in-map and it is easy to forget during review. Having it in one place in bpf_types.h will make this more obvious in my opinion. Patch 1 is an attempt to address this earlier concern in the map-in-map implementation. > 4. Then for size check change, again, it's really much simpler and > cleaner just to have a special case in check in bpf_map_meta_equal for > cases where map size matters. It may be simpler but not necessary less fragile for future map type. I am OK for removing patch 1 and just check for a specific type in patch 2 but I think it is fragile for future map type IMO. > 5. I also wonder if for those inner maps for which size doesn't > matter, maybe we should set max_elements to zero when setting > inner_meta to show that size doesn't matter? This is minor, though. > > > > Martin KaFai Lau (3): > > bpf: Clean up inner map type check > > bpf: Relax the max_entries check for inner map > > bpf: selftests: Add test for different inner map size > > > > include/linux/bpf.h | 18 +++++- > > include/linux/bpf_types.h | 64 +++++++++++-------- > > kernel/bpf/btf.c | 2 +- > > kernel/bpf/map_in_map.c | 12 ++-- > > kernel/bpf/syscall.c | 19 +++++- > > kernel/bpf/verifier.c | 2 +- > > .../selftests/bpf/prog_tests/btf_map_in_map.c | 12 ++++ > > .../selftests/bpf/progs/test_btf_map_in_map.c | 31 +++++++++ > > 8 files changed, 119 insertions(+), 41 deletions(-) > > > > -- > > 2.24.1 > >
On Thu, May 21, 2020 at 3:59 PM Martin KaFai Lau <kafai@fb.com> wrote: > > On Thu, May 21, 2020 at 03:39:10PM -0700, Andrii Nakryiko wrote: > > On Thu, May 21, 2020 at 12:18 PM Martin KaFai Lau <kafai@fb.com> wrote: > > > > > > This series allows the outer map to be updated with inner map in different > > > size as long as it is safe (meaning the max_entries is not used in the > > > verification time during prog load). > > > > > > Please see individual patch for details. > > > > > > > Few thoughts: > > > > 1. You describe WHAT, but not necessarily WHY. Can you please > > elaborate in descriptions what motivates these changes? > There are cases where people want to update a bigger size > inner map. I will update the cover letter. > > > 2. IMO, "capabilities" is word that way too strongly correlates with > > Linux capabilities framework, it's just confusing. It's also more of a > > property of a map type, than what map is capable of, but it's more > > philosophical distinction, of course :) > Sure. I can rename it to "property" > > > 3. I'm honestly not convinced that patch #1 qualifies as a clean up. I > > think one specific check for types of maps that are not compatible > > with map-in-map is just fine. Instead you are spreading this bit flags > > into a long list of maps, most of which ARE compatible. > but in one place and at the same time a new map type is added to > bpf_types.h > > > It's just hard > > to even see which ones are not compatible. I like current way better. > There are multiple cases that people forgot to exclude a new map > type from map-in-map in the first attempt and fix it up later. > > During the map-in-map implementation, this same concern was raised also > about how to better exclude future map type from map-in-map since > not all people has used map-in-map and it is easy to forget during > review. Having it in one place in bpf_types.h will make this > more obvious in my opinion. Patch 1 is an attempt to address > this earlier concern in the map-in-map implementation. Ok, just invert the condition and list only types that **are** allowed inside map-in-map. If someone forgot to add it to map-in-map check, it can be done later when someone needs it. The point is that we have a check and a list in one place, close to where it matters, instead of tracing where the value of ->capabilities comes from. Finding that in bpf_types.h is not easy and not obvious, unfortunately, and is very distant from where it's actually checked. > > > 4. Then for size check change, again, it's really much simpler and > > cleaner just to have a special case in check in bpf_map_meta_equal for > > cases where map size matters. > It may be simpler but not necessary less fragile for future map type. > > I am OK for removing patch 1 and just check for a specific > type in patch 2 but I think it is fragile for future map > type IMO. Well, if we think that the good default needs to be to check size, then similar to above, explicitly list stuff that *does not* follow the default, i.e., maps that don't want max_elements verification. My point still stands. > > > 5. I also wonder if for those inner maps for which size doesn't > > matter, maybe we should set max_elements to zero when setting > > inner_meta to show that size doesn't matter? This is minor, though. > > > > > > > Martin KaFai Lau (3): > > > bpf: Clean up inner map type check > > > bpf: Relax the max_entries check for inner map > > > bpf: selftests: Add test for different inner map size > > > > > > include/linux/bpf.h | 18 +++++- > > > include/linux/bpf_types.h | 64 +++++++++++-------- > > > kernel/bpf/btf.c | 2 +- > > > kernel/bpf/map_in_map.c | 12 ++-- > > > kernel/bpf/syscall.c | 19 +++++- > > > kernel/bpf/verifier.c | 2 +- > > > .../selftests/bpf/prog_tests/btf_map_in_map.c | 12 ++++ > > > .../selftests/bpf/progs/test_btf_map_in_map.c | 31 +++++++++ > > > 8 files changed, 119 insertions(+), 41 deletions(-) > > > > > > -- > > > 2.24.1 > > >
On Thu, May 21, 2020 at 04:10:36PM -0700, Andrii Nakryiko wrote: > > > 4. Then for size check change, again, it's really much simpler and > > > cleaner just to have a special case in check in bpf_map_meta_equal for > > > cases where map size matters. > > It may be simpler but not necessary less fragile for future map type. > > > > I am OK for removing patch 1 and just check for a specific > > type in patch 2 but I think it is fragile for future map > > type IMO. > > Well, if we think that the good default needs to be to check size, > then similar to above, explicitly list stuff that *does not* follow > the default, i.e., maps that don't want max_elements verification. My > point still stands. I think consoldating map properties in bpf_types.h is much cleaner and less error prone. I'd only like to tweak the macro in patch 1 to avoid explicit ", 0)". Can BPF_MAP_TYPE() macro stay as-is and additional macro introduced for maps with properties ? BPF_MAP_TYPE_FL() ? Or do some macro magic that the same macro can be used with 2 and 3 args?
On Thu, May 21, 2020 at 04:16:18PM -0700, Alexei Starovoitov wrote: > On Thu, May 21, 2020 at 04:10:36PM -0700, Andrii Nakryiko wrote: > > > > 4. Then for size check change, again, it's really much simpler and > > > > cleaner just to have a special case in check in bpf_map_meta_equal for > > > > cases where map size matters. > > > It may be simpler but not necessary less fragile for future map type. > > > > > > I am OK for removing patch 1 and just check for a specific > > > type in patch 2 but I think it is fragile for future map > > > type IMO. > > > > Well, if we think that the good default needs to be to check size, > > then similar to above, explicitly list stuff that *does not* follow > > the default, i.e., maps that don't want max_elements verification. My > > point still stands. > > I think consoldating map properties in bpf_types.h is much cleaner > and less error prone. > I'd only like to tweak the macro in patch 1 to avoid explicit ", 0)". > Can BPF_MAP_TYPE() macro stay as-is and additional macro introduced > for maps with properties ? BPF_MAP_TYPE_FL() ? > Or do some macro magic that the same macro can be used with 2 and 3 args? I will give it a try to minimize the code churn.
On Thu, May 21, 2020 at 4:16 PM Alexei Starovoitov <alexei.starovoitov@gmail.com> wrote: > > On Thu, May 21, 2020 at 04:10:36PM -0700, Andrii Nakryiko wrote: > > > > 4. Then for size check change, again, it's really much simpler and > > > > cleaner just to have a special case in check in bpf_map_meta_equal for > > > > cases where map size matters. > > > It may be simpler but not necessary less fragile for future map type. > > > > > > I am OK for removing patch 1 and just check for a specific > > > type in patch 2 but I think it is fragile for future map > > > type IMO. > > > > Well, if we think that the good default needs to be to check size, > > then similar to above, explicitly list stuff that *does not* follow > > the default, i.e., maps that don't want max_elements verification. My > > point still stands. > > I think consoldating map properties in bpf_types.h is much cleaner > and less error prone. Consolidation is good, but then we hopefully do it for all aspects of maps that currently have ad-hoc checks spread across a lot of places. Just looking at map_lookup_elem in syscall.c makes me wanna cry, for example :) I'll reply on another thread where Daniel proposed putting everything into ops, I like that better. > I'd only like to tweak the macro in patch 1 to avoid explicit ", 0)". > Can BPF_MAP_TYPE() macro stay as-is and additional macro introduced > for maps with properties ? BPF_MAP_TYPE_FL() ? > Or do some macro magic that the same macro can be used with 2 and 3 args?